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Degree  of  Master  of  Science  in  Aeronautics  and  Astronautics 

Abstract 

Four  predictive  ground  collision  avoidance  displays  (Break-X,  Chevrons,  Mountain,  and 
Highway)  were  tested  using  a  fixed-base  T-38  simulator  with  a  projection  screen  and  simulated 
Head-Up  Display  (HUD).  The  Break-X  was  similar  to  the  conventional  military  alert  except  that 
it  remained  on  the  screen  until  the  aircraft  was  recovered  to  safety.  The  Chevron  display 
consisted  of  two  caret  symbols  (>  <)  which  slid  horizontally  inward  as  danger  increased,  forming 
an  X  identical  to  the  break-X.  The  Mountain  display  used  a  single  icon  which  moved  upward  in 
the  HUD  as  danger  increased,  mimicking  the  motion  of  the  terrain  outside.  The  Highway  was  a 
preview  display,  which  consisted  of  a  perspective  elevated  surface  shown  at  the  desired  altitude. 

Twelve  subjects  flew  through  a  series  of  Predictive  Ground  Collision  Avoidance  System 
(PGCAS)  situations  in  zero  visibility  using  each  display.  For  about  30  seconds  after  the  alert, 
they  attempted  to  maintain  a  set  altitude  above  ground,  clear  of  the  terrain  but  below  ground 
radar  threats.  Performance  metrics  were  rolling  tendency,  altitude  maintenance,  pilot  effort,  and 
subjective  preference. 

The  Break-X  performed  more  poorly  than  the  other  displays  in  every  category.  It  attracted 
attention  but  proved  to  be  impractical  for  the  terrain-following/terrain-avoidance  task.  Pilots 
were  able  to  spend  only  40%  of  the  flight  time  between  the  desired  altitudes  when  given  the 
Break-X,  and  on  average,  they  crashed  every  5  runs.  The  Chevrons  were  more  useful,  although 
their  horizontal  motion  did  not  correspond  to  the  outside  world.  The  Chevrons  and  Mountain 
averaged  only  one  crash  in  12  runs.  The  vertically  moving  Mountain  had  physical  analogue,  so 
pilots  found  it  more  natural  to  follow.  This  enabled  them  to  spend  approximately  80%  of  the 
time  between  the  altitude  limits,  while  the  Chevrons  allowed  70%.  The  better  altitude 
performance  from  the  Mountain  came  at  the  cost  of  higher  effort  levels,  as  shown  by  a 
significant  difference  in  RMS  longitudinal  stick  movement.  However,  a  false  illusion  of  wings- 
level  produced  slightly  poorer  roll  performance  from  the  Mountain.  Pilots  crashed  the  least  using 
the  Highway,  averaging  about  one  crash  per  50  runs.  It  enabled  them  to  fly  approximately  90% 
of  the  time  within  the  desired  altitude  layer.  The  Highway  produced  the  highest  objective  and 
subjective  ranking,  and  its  predictive  nature  made  it  the  best  display  for  the  task  investigated. 

Thesis  Supervisor;  James  K.  Kuchar 

Assistant  Professor  of  Aeronautics  and  Astronautics 
Technical  Advisor:  Steven  W.  Jacobson 

Charles  Stark  Draper  Laboratory 
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Abstract 

Four  predictive  ground  collision  avoidance  displays  (Break-X,  Chevrons,  Mountain,  and 
Highway)  were  tested  using  a  fixed-base  T-38  simulator  with  a  projection  screen  and  simulated 
Head-Up  Display  (HUD).  The  Break-X  was  similar  to  the  conventional  military  alert  except  that 
it  remained  on  the  screen  until  the  aircraft  was  recovered  to  safety.  The  Chevron  display 
consisted  of  two  caret  symbols  (>  <)  which  slid  horizontally  inward  as  danger  increased,  forming 
an  X  identical  to  the  break-X.  The  Mountain  display  used  a  single  icon  which  moved  upward  in 
the  HUD  as  danger  increased,  mimicking  the  motion  of  the  terrain  outside.  The  Highway  was  a 
preview  display,  which  consisted  of  a  perspective  elevated  surface  shown  at  the  desired  altitude. 

Twelve  subjects  flew  through  a  series  of  Predictive  Ground  Collision  Avoidance  System 
(PGCAS)  situations  in  zero  visibility  using  each  display.  For  about  30  seconds  after  the  alert, 
they  attempted  to  maintain  a  set  altitude  above  ground,  clear  of  the  terrain  but  below  ground 
radar  threats.  Performance  metrics  were  rolling  tendency,  altitude  maintenance,  pilot  effort,  and 
subjective  preference. 

The  Break-X  performed  more  poorly  than  the  other  displays  in  every  category.  It  attracted 
attention  but  proved  to  be  impractical  for  the  terrain-following/terrain-avoidance  task.  Pilots 
were  able  to  spend  only  40%  of  the  flight  time  between  the  desired  altitudes  when  given  the 
Break-X,  and  on  average,  they  crashed  every  5  runs.  The  Chevrons  were  more  useful,  although 
their  horizontal  motion  did  not  correspond  to  the  outside  world.  The  Chevrons  and  Mountain 
averaged  only  one  crash  in  12  runs.  The  vertically  moving  Mountain  had  physical  analogue,  so 
pilots  found  it  more  natural  to  follow.  This  enabled  them  to  spend  approximately  80%  of  the 
time  between  the  altitude  limits,  while  the  Chevrons  allowed  70%.  The  better  altitude 
performance  from  the  Mountain  came  at  the  cost  of  higher  effort  levels,  as  shown  by  a 
significant  difference  in  RMS  longitudinal  stick  movement.  However,  a  false  illusion  of  wings- 
level  produced  slightly  poorer  roll  performance  from  the  Mountain.  Pilots  crashed  the  least  using 
the  Highway,  averaging  about  one  crash  per  50  runs.  It  enabled  them  to  fly  approximately  90% 
of  the  time  within  the  desired  altitude  layer.  The  Highway  produced  the  highest  objective  and 
subjective  ranking,  and  its  predictive  nature  made  it  the  best  display  for  the  task  investigated. 
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1  Introduction 


1.1  Problem  Motivation 

Controlled  Flight  Into  Terrain  (CFIT)  is  a  major  problem  for  today’s  low-flying  military 
aircraft.  In  CFTT  accidents,  the  aircraft  is  controllable  and  has  sufficient  energy  to  clear  the 
terrain  ahead,  but  the  pilot’s  Situational  Awareness  (SA)  is  degraded  to  the  point  where  a 
collision  occurs.  The  degradation  may  involve  limited  visibility,  cognitive  tunneling,  over¬ 
tasking,  under-tasking,  or  a  lack  of  altitude  and/or  attitude  cues.  CFTT  is  partially  combated  by 
Ground  Collision  Avoidance  Systems  (GCAS),  known  in  the  civilian  sector  as  Ground  Proximity 
Warning  Systems  (GPWS).  GCAS  warns  the  pilot  just  before  the  collision  would  occur,  with  the 
intention  to  allow  a  successful  recovery. 

A  problem  can  occur,  though,  when  visibility  is  limited  and  the  pUot  does  not  recover  the 
aircraft  to  a  reasonable  altitude  after  receiving  the  alert.  He  may  not  pull  hard  enough  or  long 
enough,  resulting  in  a  CFTT  accident.  Alternatively,  he  may  pull  too  long,  unnecessarily 
exposing  the  airplane  to  ground  radar  and  wasting  valuable  time  and  energy.  In  either  of  these 
two  cases,  the  pilot  does  not  have  sufficient  information  about  the  terrain  to  plan  a  conservative, 
safe  recovery.  Part  of  the  reason  behind  this  may  be  the  current  standard  visual  alert  provided  on 
both  the  Head-Down  Displays  (HDD)  and  the  Head-Up  Display  (HUD).  When  the  GCAS  issues 
an  alert,  the  pilot  hears  an  aural  message  such  as,  “Pull-Up!  Pull-Up!”  while  an  X  symbol  flashes 
on  the  HUD.  The  X  then  disappears  after  2-5  seconds,  leaving  the  pilot  to  his  own  devices  to 
make  the  recovery. 

The  purpose  of  this  thesis  is  to  investigate  several  visual  GCAS  displays,  which  give  the 
pilot  additional  guidance  after  the  initial  warning  has  been  received.  Specifically,  it  compares 
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their  ability  to  help  the  pilot  recover  to  a  safe  altitude;  clear  of  the  terrain  but  low  enough  to 
preserve  survivability  in  the  presence  of  ground  threats. 

1.2  Thesis  Summary 

Chapter  2  provides  background  on  CFIT,  including  the  magnitude  of  the  problem  and 
causes  behind  CFIT  accidents.  It  then  details  the  advantages  of  digital  terrain  databases  and 
discusses  the  theory  behind  and  the  implementation  of  Predictive  GCAS  (PGCAS).  The 
shortfalls  of  current  military  PGCAS  symbology  are  talked  about  at  the  end  of  the  chapter. 

In  Chapter  3,  several  ideas  are  discussed  as  possible  solutions  to  the  symbology 
shortfalls.  In  particular,  the  section  talks  about  dynamic  PGCAS  displays,  which  provide  more 
information  and  better  feedback  to  the  pilot  during  a  PGCAS -prompted  recovery.  These  displays 
are  alternatives  to  the  break-X  commonly  used  in  most  military  aircraft.  They  include 
horizontally  and  vertically  moving  displays  along  with  perspective  symbology,  which  shows 
multiple  terrain  elevations  to  the  pilot.  The  hypothesis  is  given  at  the  end  of  Chapter  3. 

Chapter  4  detaUs  the  experiment  used  to  test  four  candidate  PGCAS  displays,  which  fall 
into  the  categories  mentioned  in  the  previous  chapter.  The  simulated  ground  collision  avoidance 
algorithm  needed  to  drive  the  displays  is  discussed  first,  foUowed  by  the  symbologies 
themselves.  The  next  section  explains  the  hardware  and  software  used  to  implement  the 
simulation,  and  the  last  two  sections  review  the  simulation  and  the  experimental  design. 

The  results  are  presented  in  Chapter  5,  which  includes  the  subject  demographics.  The 
chapter  quantifies  the  performance  of  the  four  displays  during  each  phase  of  recovery.  Subject 
comments  are  also  included  as  a  rich  source  of  subjective  data.  The  Anal)4ical  Hierarchy  Process 
used  to  analyze  the  numerical  questionnaire  data  is  laid  out  in  the  last  section. 
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The  sixth  and  final  chapter  gives  conclusions  and  recommendations  drawn  from  the 
results  of  Chapter  5.  Included  are  thoughts  to  be  drawn  from  this  research  and  further  research 
that  should  be  performed  in  the  area  of  GCAS  symbology. 
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2  Background 


2.1  Controlled  Flight  Into  Terrain 

Controlled  flight  into  terrain  (CFTT)  is  a  major  cause  of  aviation  accidents  today.  It 
occurs  when  the  pilot  unknowingly  flies  a  controllable  aircraft  into  the  ground,  generally  due  to  a 
lack  of  situational  awareness  (SA).  In  these  situations,  the  aircraft  typically  has  sufficient  energy 
and  control  authority  to  avoid  the  collision.  Presenting  terrain  proximity  information  to  the  pilot 
in  a  concise,  understandable  way  could  prevent  many  of  these  accidents. 

2.1.1  Problem  Magnitude 

CFTT  accidents  still  occur  in  both  civilian  and  military  aviation,  although  the  rates  have 
been  greatly  reduced  in  recent  years.’  This  is  mainly  due  to  federally  mandated  Ground 
Proximity  Warning  Systems  (GPWS)  and  their  military  equivalent.  Ground  Collision  Avoidance 
Systems  (GCAS).  Other  improvements  include  better  Air  Traffic  Control  (ATC)  radar  coverage. 
Minimum  Safe  Altitude  Warning  systems,  and  better  approach  aids.  Pilot  education  and 
simulator  training  have  also  contributed  to  the  decrease.  Still,  an  average  of  31  turbojet  and 
propeller  aircraft  were  lost  to  CFTT  each  year  fi-om  1983-1992.  ^  In  the  past  20  years,  over  half  of 
the  fatalities  in  airline  crashes  have  been  caused  by  CFIT.^  The  problem  has  by  no  means  gone 
away,  and  today  passenger  aircraft  crash  at  the  rate  of  about  one  per  year  due  to  CFTT. 

In  military  flying,  some  of  the  systems  that  have  reduced  civilian  CFTT  do  not  apply. 
Many  missions  are  flown  outside  or  underneath  ATC  radar  coverage.  Military  pilots  have 
motivation  in  many  cases  to  stay  near  the  terrain,  not  well  above  it.  This  flying  is  done  in  all 
weather  conditions,  at  any  time  of  day,  and  often  these  factors  combine  to  cause  CFTT  accidents. 

Between  FY  1980  and  FY  1997,  CFIT  was  a  causal  factor  in  176  of  945  total  USAF 
Class  A  mishaps  (almost  20%).'’  Class  A  mishaps  are  defined  by  complete  loss  of  an  aircraft,  a 
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fatality,  or  more  than  $1  million  in  damage.^  CFIT  accidents  cost  the  Air  Force  an  average  of 
$181  million  annually  during  this  period.  Many  are  fatal,  and  170  of  the  452  Air  Force  pilots 
killed  during  that  time  period  died  in  CHT  crashes  (almost  40%).  The  Air  Force  is  not  the  only 
service  plagued  by  CFIT.  Between  FY  1983  and  FY  1997,  the  Navy  and  Marine  Corps 
experienced  106  and  62  Class  A  CFIT  accidents  respectively.^  This  is  a  very  real  problem,  and 
innovation  is  needed  in  the  field  to  force  these  rates  down. 

2.1.2  CFIT  Causes 

In  a  CFIT  collision,  the  aircraft  impacts  the  terrain  because  the  pilot  is  unaware  of  the 
aircraft’s  altitude  or  the  increasing  terrain  altitude.  This  SA  deficit  sometimes  arises  due  to  a 
pilot  workload  that  is  too  heavy  or  too  light.  The  pilot  may  be  fixated  outside  the  aircraft, 
attempting  to  target  another  aircraft  or  an  object  on  the  ground.  Examples  are  watching  the  bomb 
fall  line  and  other  targeting  symbology  just  before  weapons  release,  or  checking  visually  to  see 
whether  those  bombs  hit  the  target.^  Alternatively,  the  pilot  might  be  occupied  with  something 
inside  the  aircraft  such  as  a  checklist  or  procedure.  In  both  of  these  cases,  the  pilot’s  attention  is 
devoted  to  something  other  than  safe  control  of  the  aircraft.  This  is  termed  cognitive  capture 
because  the  pilot  no  longer  switches  efficiently  between  the  outside  world  and  the  aircraft  or 
between  different  indicators  in  the  aircraft.*  This  is  especially  applicable  in  military  aviation, 
which  includes  maneuvering  at  low  altitudes  to  avoid  threat  detection  and  operational  tactics 
requiring  high  ground  closure  rates.^  Tasks  that  commonly  preoccupy  the  pilot  are  navigation, 
attack  planning,  threat  evasion,  and  target  acquisition  at  extremely  low  altitudes.  Of  the  59 
Class- A  mishaps  in  the  F-16  between  1980  and  1989,  53%  occurred  during  low-level  or 
maneuvering  flight,  and  60%  were  likely  due  to  channelized  attention.^ 
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Another  opportunity  for  CFIT  is  when  the  pilot  lacks  information  about  the  aircraft’s 
position.  The  aircraft  slowly  descends  until  it  reaches  the  ground,  or  it  remains  at  a  constant 
altitude  while  the  terrain  below  rises  to  meet  it.  This  happens  in  environments  lacking  altitude 
cues.^  Terrain  devoid  of  features,  like  a  snowy  plain  or  sandy  desert,  gives  little  altitude 
information.  Right  over  the  ocean  sometimes  leads  to  CRT  because  the  colors  of  water  and  sky 
can  be  quite  similar,  making  the  horizon  almost  invisible.  Darkness,  fog,  smoke,  and  haze  also 
eliminate  many  of  the  visual  stimuli  that  would  otherwise  help  the  pilot  remain  aware  of 
altitude.’  Poor  weather  is  a  major  cause,  especially  in  military  flying  where  aircraft  are  more 
likely  to  be  near  the  ground  when  flying  through  clouds  and  fog. 

2.2  Ground  Collision  Avoidance  Systems 

CRT  is  partially  mitigated  through  the  use  of  military  GCAS  and  the  civilian  equivalent 
termed  GPWS.  These  systems  were  first  fitted  to  jet  transport  aircraft  in  the  mid-1970’s  and  have 
improved  greatly  since  then.^  Originally,  GCAS  simply  warned  the  pilot  when  the  altitude  above 
ground  level  (AGL),  determined  by  the  radar  altimeter,  went  below  a  set  threshold.  Today, 
Predictive  GCAS  (PGCAS)  and  Enhanced  GPWS  (EGPWS)  incorporate  position,  velocity,  turn 
rate,  and  other  factors  to  give  a  more  accurate  warning.  Because  PGCAS  and  EGPWS  include 
predictive  information,  they  are  even  more  effective  than  their  first-generation  predecessors. 

Based  on  current  position  and  velocity,  PGCAS  projects  a  recovery  maneuver  that 
accounts  for  pilot  and  aircraft  reaction  time  and  a  maximum  performance  pull-up.  This  is 
compared  to  a  worst-case  terrain  profile  based  on  the  highest  terrain  the  aircraft  could  feasibly 
encounter.  Often,  the  terrain  altitude  ahead  is  extrapolated  linearly  using  the  radar  altimeter.  This 
works  well  in  terrain  that  has  zero  or  constant  gradient,  but  in  rugged  hills  and  mountains  the 
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performance  of  these  systems  quickly  degrades.  Alternatively,  the  profile  comes  from  a  digital 
terrain  database  stored  onboard  the  aircraft. 

2.2.1  Digital  Terrain 

In  the  past,  the  only  good  sources  of  terrain  information  were  terrain-following  radar 
(TFR)  and  the  radar  altimeter.  Both  of  these,  however,  have  limitations.  The  radar  altimeter  gives 
only  current  information  and  cannot  determine  the  terrain  height  ahead  of  the  aircraft.  Most  are 
also  roll-limited  since  the  radar  sensor  must  be  pointing  toward  the  ground  for  the  instrument  to 
function.  TFR  is  relatively  expensive,  consumes  space,  and  is  limited  by  field-of-view.  There  are 
also  time  delays  associated  with  slewing  the  radar  and  data  processing.^  Today,  increased 
computational  power  in  modern  on-board  flight  computers  and  inexpensive  flash  memory  have 
made  possible  an  alternative:  the  use  of  digital  terrain  elevation  data.^  These  data  are  generally 
taken  from  satellite  surveys  which  are  digitized  at  a  set  resolution  to  produce  terrain  maps.  The 
maps  include  a  set  of  latitude-longitude  coordinates  with  a  maximum  terrain  altitude  at  each 
location.  Some  even  incorporate  man-made  obstacles  like  towers,  antennae,  bridges,  and 
suspended  cables.”  The  most  readily  available  set  of  terrain  data  is  the  Digital  Terrain  Elevation 
Data  (DTED)  produced  by  the  National  Imagery  and  Mapping  Association  (NIMA).  Generally, 
closer  post  spacing  (better  resolution)  must  be  traded  against  coverage  since  the  highest  quality 
data  are  not  available  for  the  entire  earth.  Table  2. 1  shows  the  DTED  levels  along  with  spacing 
and  expected  worldwide  coverage. 
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Table  2.1:  DTED  Summary^^ 


DTED  Level 

Nominal  Post  Spacing  (m) 

Expected  Coverage 
(2001) 

0 

1000 

100% 

1 

100 

85-90% 

2 

30 

80% 

DTED  is  useful  for  terrain-referenced  navigation,  obstacle  warnings,  and  passive  ranging. 
The  digital  terrain  data  can  also  be  used  to  improve  PGCAS  systems.  If  the  flight  computer  has 
an  accurate  altitude  profile  of  the  terrain  which  the  aircraft  will  soon  fly  over,  look-ahead  alerting 
can  determine  more  accurately  whether  the  aircraft  is  in  danger  of  impacting  the  ground.  These 
systems  are  sometimes  termed  Passive  PGCAS  (PPGCAS)  because  they  do  not  rely  on  active 
sensors  for  terrain  information,  but  the  acronym  is  often  shortened  to  PGCAS.  Better  terrain 
knowledge  reduces  the  number  of  nuisance  alerts,  allowing  pilots  to  trust  more  faithfully  in  their 
GCAS.  Predictive  Ground  Collision  Avoidance  Systems  can  look  over  the  horizon  to  provide 
earlier  alerts,  work  well  in  turning  flight,  and  are  less  expensive  than  active  systems  once  the 
terrain  database  is  established.*^ 

2.2.2  PGCAS  Algorithm 

Generally,  the  PGCAS  algorithm  searches  the  database  for  terrain  locations  within  a 
polygon  ahead  of  the  aircraft.  The  size  and  shape  of  the  polygon  may  change  based  on  the 
current  aircraft  state.  For  example,  if  the  aircraft  is  in  a  right  turn,  the  polygon  can  be  expanded 
to  the  right  since  the  terrain  there  is  more  likely  to  be  encountered.  Figure  2.1  shows  typical 
shapes  for  the  terrain  search  polygon  in  both  straight  and  turning  flight.*^’*^  This  terrain  search 
pattern  is  similar  to  that  used  by  British  Aerospace  in  their  Digital  Terrain  System  (DTS).*^ 
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Figure  2.1;  Terrain  Search  Polygons 

The  terrain  posts  that  fall  inside  this  polygon  are  analyzed  using  binning  and  hulling.*^  Binning 
involves  a  radial  sweep  across  the  polygon  to  find  the  highest  post  at  each  of  a  series  of  distances 
from  the  aircraft.  The  range  bin  size  (RBS)  is  the  distance  increment  at  which  these  sweeps  take 
place,  and  its  choice  involves  trading  accuracy  for  throughput.  Making  the  range  bins  smaller 
means  that  less  error  will  be  made  when  they  are  hulled  together.  Binning  compresses  the 
polygon  into  a  worst-case  two-dimensional  profile  of  highest  terrain  points.  Hulling  is  the 
process  in  which  the  points  are  connected  to  estimate  the  terrain  elevation  between  the  posts.  The 
hulls  between  the  terrain  posts  are  always  convex  to  make  the  estimated  terrain  height 
conservative.  Figure  2.2  shows  the  binning  and  hulling  processes. 
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Largest  elevation 


speed,  configuration,  and  weight  to  make  the  algorithm  even  more  accurate.  The  pull-up  is 
generally  calculated  using  a  constant  radius  based  on  the  load  factor,  and  the  aircraft  is  expected 
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to  climb  straight  ahead  after  reaching  some  flight  path  angle.  Figure  2.3  shows  the  projected 
recovery. 


Figure  2.3:  Projected  Recovery 

The  warning  is  issued  when  the  predicted  trajectory  comes  too  close  to  the  terrain  hull.  The 
minimum  distance  between  the  terrain  profile  and  the  projected  recovery  is  the  parameter  used 
by  the  alerting  algorithm.  An  alert  is  issued  when  the  distance  no  longer  exceeds  a  safety 
clearance  height  set  by  the  pilot.  The  safety  clearance  height  is  a  function  of  the  terrain  type,  area 
threats,  aircraft  performance,  mission  length,  and  other  factors. 

2.2.3  Current  PGCAS  Symbology 

When  a  PGCAS  call  is  received,  both  audio  and  visual  cues  inform  the  pilot.  Usually,  a 
synthesized  voice  in  the  pilot’s  headset  exclaims,  “Pull  up!  Pull  up!”  or  “Altitude!  Altitude!” 
Simultaneously,  a  unique  symbol  appears  on  the  head-up  display  (HUD).  The  standard  military 
PGCAS  symbol,  shown  in  Figure  2.4,  is  termed  the  “breakaway  cue”  or  “break-X.”*® 
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Figure  2.4;  Break-X 

It  consists  of  two  lines  meeting  at  right  angles  to  form  an  X,  and  it  occults  all  other  HUD 
symbology  in  most  cases.  The  size  of  the  symbol  varies  between  aircraft,  but  the  standard  is 
about  100  mrad.  The  symbol  flashes  at  a  rate  of  3-10  Hz,  and  it  disappears  completely  after  2-5 
seconds.  It  is  usually  located  at  the  center  of  the  pilot  field-of-view  (FOV)  or  in  some  cases  at  the 
flight  path  marker  (FPM).  The  break-X  is  visible  in  most  or  all  of  the  available  HUD  modes. 
This  alert  is  used  in  the  A- 10,  the  F-16,  and  other  fast-moving  jets  in  the  attack  and  fighter 
classes.  When  the  pilot  receives  the  break-X,  training  takes  over.  He  or  she  knows  instinctively 
that  danger  is  present,  and  reacts  by  rolling  to  wings-level  and  pulling.  Although  pilots 
sometimes  use  a  loaded  roll  (pulling  while  still  banked),  minimum  altitude  loss  is  achieved  by 
first  rolling  to  wings-level.  The  terrain  features  sometimes  make  a  loaded  roll  more 
advantageous,  especially  when  low  terrain  is  seen  to  one  side  or  the  other.  The  pilot  must  then 
decide  how  hard  to  pull  and  how  long  to  continue  the  recovery.  The  pilot  uses  a  g-onset  rate 
specified  by  the  flight  manual  and  pulls  to  maximum  performance.  Some  aircraft  provide  natural 
cues  such  as  buffeting  when  maximum  performance  is  reached.  They  may  also  incorporate  stick 
shakers  or  stall  horns,  which  vary  in  fi-equency  or  magnitude  to  show  the  stall  progression.  In  the 
A- 10,  which  has  few  natural  stall  indications,  maximum  performance  is  found  by  pulling  until 
the  stall  warning  horn  is  “between  the  steady  and  chopped  tones.”^^  In  any  case,  the  pilot  pulls 


up  and  then  levels  off  or  continues  to  climb  based  on  the  perceived  terrain  proximity  and  other 
factors. 

2.3  Symbology  Shortfalls 

One  problem  with  existing  PGCAS  is  the  fact  that  civilian  and  military  pilots  cannot 
completely  rely  on  the  system.  When  the  alert  occurs,  pilots  sometimes  attempt  to  verify  it 
before  recovering  by  looking  for  other  sources  of  altitude  information  (radar  and  barometric 
altimeter,  visual  scene).  This  period  of  time  may  be  enough  for  the  airplane  to  impact  the  ground, 
while  an  immediate  recovery  would  have  prevented  the  CRT.  This  “delayed  response 
syndrome”  is  partly  caused  by  previously-experienced  nuisance  alerts,  which  desensitize  pilots  to 
the  PGCAS.  But  it  also  may  be  contributed  to  by  the  on-off  nature  of  the  break-X.  Because  the 
symbol  looks  the  same  in  any  warning  situation,  a  single  bump  in  the  terrain  produces  the  same 
alert  as  a  towering  mountain  peak  ahead  of  the  aircraft.  If  there  were  some  way  of  putting  the 
alert  in  context,  such  as  a  variable-intensity  alert,  the  pilots  would  see  not  just  the  final  warning 
but  also  the  increasingly  dangerous  situation  that  led  to  the  warning.  Hopefully,  they  would  be 
able  to  initiate  the  recovery  immediately  because  they  have  already  evaluated  the  situation. 

The  break-X  originated  while  the  radar  altimeter  was  still  the  only  source  of  PGCAS 
information,  and  since  that  time  the  display  requirements  have  changed.  And  because  more 
information  is  available  for  presentation,  other  symbols  should  be  considered  which  would  help 
the  pilot  in  more  situations.  In  a  study  of  automatic  and  manual  PGCAS  recoveries,  researchers 
at  the  Naval  Test  Center  concluded  that  “Pilots  rely  heavily  upon  [visual  PGCAS]  cues  to 
perform  maneuvers  such  as  a  constant  g  pull-up  to  wings-level . . .  The  more  cues  provided  to  the 
pilots  about  impending  trouble  ...  the  better. Modem  PGCAS  algorithms  discussed  above 
already  utilize  terrain  elevations  at  multiple  distances  firom  the  aircraft.  It  is  possible  to  show 
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more  of  this  information  to  the  pilot  without  excessively  cluttering  the  HUD.  Benefits  would 
include  more  confident  low-level  operation  in  deteriorating  weather  conditions,  smoother 
PCKIAS  recoveries,  and  appropriate  level-offs  that  preserve  terrain  masking. 

2.3.1  Low-Level  Operation 

In  many  cases,  military  pilots  perform  low-level  weapons  delivery  in  marginal  or  poor 
weather  conditions.  They  may  be  flying  aircraft  not  equipped  with  a  terrain-following  (TF) 
system;  in  other  cases  TF  is  available  but  inappropriate.  Consider  the  situation  shown  in  Figure 
2.5  where  a  gap  exists  between  the  terrain  and  the  clouds  that  a  pilot  must  penetrate  enroute  to 
the  target.  The  problem  is  that  he  does  not  know  what  the  weather  or  terrain  looks  like  beyond 
that  gap.  He  knows  the  gap  is  penetrable  but  feels  unsafe  flying  through  it,  knowing  that  the 
ceiling  may  be  down  to  the  surface  on  the  other  side. 


Figure  2.5:  Penetration  Gap 


If  the  pilot  could  rely  on  the  PGCAS  for  not  just  a  last-minute  warning  but  also  real-time 
guidance  through  the  terrain,  he  could  confidently  continue  the  mission.  This  would  allow  a 
weapons  delivery  in  a  situation  where  otherwise  an  abort  might  have  occurred. 

2.3.2  Smooth  Recoveries 

Leveling  off  after  the  recovery  and  wondering  if  another  alert  will  occur  gives  an 
insecure  feeling  and  adds  to  the  already  stressfiil  situation.  If  the  pilot  wants  to  recover  but 
remain  fairly  near  the  ground,  he  must  weigh  his  safety  against  the  chance  that  another  alert  will 


29 


occur.  If  the  break-X  is  to  be  used  for  recovering  to  an  appropriate  altitude  and  resuming  the 
mission,  the  pilot  must  oscillate  about  the  altitude  where  the  break-X  appears.  This  takes  a  large 
amount  of  pilot  effort  and  is  very  dangerous  because  no  indication  is  given  until  danger  is 

already  present.  This  makes  the  break-X  impractical  if  not  useless  for  staying  above  but 
somewhat  close  to  the  terrain 

2.3.3  Appropriate  Recoveries 

The  typical  break-X  gives  no  visual  indication  when  the  aircraft  is  sufficiently  recovered. 
The  pilot  would  rather  err  to  the  conservative  side,  especially  in  instrument  conditions.  This 
results  in  large  altitude  gains  in  some  cases,  excessive  in  others.  In  cases  where  such  a  maneuver 
IS  really  needed  to  avoid  the  terrain,  it  is  appropriate.  But  in  many  situations,  such  aggressive 
action  is  unnecessary  and  even  dangerous.  Large  altitude  gains  unnecessarily  expose  the  aircraft, 
waste  energy,  use  up  valuable  time,  and  disrupt  missions.  It  is  generally  agreed  that  the  hard 
pull-up  is  “not  a  manoevre  to  be  encouraged  in  the  vicinity  of  enemy  air  defenses.”^^  Consider  a 
PGCAS  alert  received  because  the  aircraft  descended  slightly  too  low  while  crossing  a  ridge.  The 
resulting  recovery,  shown  in  Figure  2.6,  is  almost  completely  unneeded  and  places  the  aircraft  in 
danger  of  being  targeted  by  the  surface-to-air  missile  (SAM). 


Figure  2.6:  Ridge  Clip  Recovery 
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One  can  see  that  the  aircraft  recovers  into  the  SAM  coverage  area  where  a  shootdown  might 
occur.  The  pilot  really  has  no  other  option,  though,  because  a  collision  with  the  terrain  would  be 
even  more  devastating.  Generally,  it  is  uncomfortable  to  descend  in  instrument  conditions  when 
one  has  just  received  a  PGCAS  call.  In  some  situations,  though,  this  may  be  necessary  to 
preserve  the  stealth  of  the  mission.  In  another  situation,  shown  in  Figure  2.7,  an  aggressive 
recovery  is  needed  to  clear  the  terrain. 


Figure  2.7:  Insufficient  Recovery 

The  pilot  pulls  hard  to  avoid  the  first  ridge  but  leaves  the  throttle  at  less  than  fuU  power  because 
he  cannot  see  the  second  ridge.  Airspeed  is  lost  during  this  pull-up,  and  when  the  second  alert  is 
received,  the  aircraft  has  too  little  energy  to  pull  up  and  avoid  the  ground.  If  the  pilot  would  have 
had  an  earlier  indication  that  the  second  ridge  was  present,  he  could  have  used  full  power  upon 
receiving  the  first  alert  and  cleared  both  ridges.  This  type  of  CFIT  accident  would  occur  more 
often  with  a  climb-limited  aircraft  like  the  A-10  because  the  pilot  must  make  power  adjustments 
early  to  conserve  energy.  Because  the  break-X  is  binary,  it  gives  no  information  until  danger  is 
already  present.  It  also  gives  information  about  the  terrain  at  only  one  distance  ahead  of  the 
aircraft,  making  it  impossible  to  show  the  pilot  both  ridges  in  the  previous  scenario. 

In  the  situations  mentioned  above,  the  break-X  serves  its  originally  intended  purpose.  It 
makes  the  pilot  aware  of  the  terrain  and  allows  him  sufficient  time  to  react,  roll,  and  pull.  The 
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problem  is  that  no  indication  is  given  of  the  severity  of  the  situation  or  whether  the  aircraft  is  no 
longer  in  danger  of  ground  impact.  A  visual  display  should  be  considered  to  replace  the  break-X 
that  provides  more  guidance  to  the  pilot  during  and  after  the  recovery.  When  the  break-X  was 
first  implemented,  the  only  information  available  fi-om  the  PGCAS  was  that  a  recovery  must  be 
initiated.  With  predictive  systems  and  digital  terrain  data,  however,  the  system  is  much 
“smarter.”  It  knows  not  just  that  high  terrain  exists,  but  also  that  terrain’s  distance  and  direction 
from  the  aircraft.  It  also  knows  what  other  ground  features  exist  beyond  the  nearest  terrain  that 
may  prove  to  be  dangerous.  A  good  PGCAS  display  would  give  the  pilot  some  of  this 
information,  enabling  more  accurate,  confident  flying.  Current  research  is  investigating 
automatic  PGCAS,  which  actually  implements  the  recovery  by  taking  control  of  the  aircraft.  This 
is  not  possible  on  many  currently  fielded  aircraft,  while  changing  PGCAS  symbology  is  possible. 
Also,  passive  systems  enable  the  pilot  to  maintain  full  control  of  the  aircraft,  which  is  desired  in 
most  circumstances  if  the  pilot  is  adequately  informed. 
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3  Advanced  PGCAS  Displays 


Visual  symbologies  should  be  investigated  which  show  the  pilot  a  safe  recovery  path  or 
command  him  through  one.  The  simplest  of  these  recoveries  would  be  a  hard  pull  followed  by  a 
level-off  at  some  safe  altitude  above  the  terrain.  The  altitude  should  be  high  enough  that  the 
PGCAS  will  not  immediately  trigger  another  alert,  but  it  must  be  low  enough  that  the  aircraft 
remains  within  the  desired  constraints  to  avoid  radar  coverage.  This  altitude  might  change 
depending  on  the  distance  from  the  nearest  SAM  site.  The  display  should  not  leave  the  pilot  with 
an  uncomfortable  feeling  when  leveling  off  or  descending,  wondering  whether  he  will  impact  the 
terrain.  One  improvement  is  to  have  an  unobtrusive  alert  that  remains  on  the  HUD  until  the  pilot 
physically  acknowledges  it  by  pushing  a  button  or  simply  maneuvering  to  a  safer  altitude.  Pilot 
feedback  in  the  C-130  DTS  project  indicated  that  most  pilots  would  like  a  consent/disregard 
button  that  would  “indicate  a  positive  pilot  response  for  each  [Passive  PGCAS]  warning.”  Pilot 
comments  in  the  development  of  KC-135  PGCAS  indicated  that  “the  GCAS  warning  should  be 
present  for  as  long  as  the  warning  conditions  exist  or  until  the  pilot  activates  a  GCAS  system 
reset  switch.”^^  This  would  allow  the  pilot  to  decide  when  enough  guidance  has  been  received. 
This  approach  is  more  readily  accepted  by  pilots  of  heavy  aircraft  because  they  are  generally  less 
task-loaded,  and  the  crew  of  two  has  more  time  to  positively  acknowledge  the  alert.  Even  in  fast- 
movers,  though,  the  additional  guidance  and  comfort  provided  by  such  a  system  might  outweigh 
the  nuisance  of  pilot-acknowledgement. 

3.1  Possible  Solutions 

3.1.1  Horizontally  Moving  Displays 

If  the  PGCAS  alert  is  allowed  to  remain  on  the  HUD  until  the  pilot  resets  the  system  with 
a  switch  or  recovers  to  safety,  alternatives  to  the  break-X  would  provide  better  guidance  during 
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the  recovery.  The  first  step  in  improving  the  alert  might  be  to  incorporate  rate  information  to 
improve  the  binary  nature  of  the  break-X.  Since  the  X-shape  is  commonly  recognized  as  a 

PGCAS  symbol,  rate  information  could  be  added  by  splitting  the  X  into  two  chevrons  that  move 
horizontally  on  the  HUD  (Figure  3.1). 


Figure  3.1:  Chevrons 


The  pilot  would  see  the  two  chevrons  approaching  one  another  until  they  form  an  X  in  the  center 
of  the  HUD.  This  provides  precursory  warning  information  to  the  pilot,  allowing  him  to  examine 
the  situation  before  the  X  appears,  demanding  immediate  recovery.  From  the  speed  at  which  the 
chevrons  come  together,  the  pilot  can  also  estimate  his  terrain-closure  rate.  A  gain  must  be 
chosen  so  that  the  chevron  movement  is  slow  enough  to  be  useful  to  the  pilot  but  fast  enough  to 
provide  plenty  of  advance  warning.  Pilots  who  flew  an  F/A-18  equipped  with  a  chevron  PGCAS 
display  agreed  that  “...  the  chevrons  provided  several  necessary  cues  ...  that  could  be  absorbed 
through  their  peripheral  view.  For  instance,  the  rate  of  closure  provided  an  easily  perceptible  cue 
on  the  sense  of  urgency  of  the  situation.”^*^  Finally,  the  pilot  sees  the  situation  improve  as  he 
recovers.  As  he  pulls,  the  chevrons  slow  their  inward  progress  and  begin  to  separate,  providing 
positive  feedback.  The  F/A-18  research  noted  that  “Slowing  of  the  closure  rate  or  chevron 
separation  provided  useful  feedback  on  if  and  how  well  the  pilot  was  affecting  the  situation  when 
he  chose  to  intervene.”^®  To  remain  somewhat  near  the  terrain  after  the  initial  recovery  while 
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ensuring  his  safety,  the  pilot  could  fly  so  that  the  chevrons  maintain  some  horizontal  distance 
from  center-HUD. 

A  chevron  display  is  currently  being  used  in  flight  testing  of  the  Advanced  Fighter 
Technology  Integration  (AFTI)  F-16  Automatic  GCAS.^^  The  distance  between  the  chevrons  is 
based  on  the  time  until  the  Auto-GCAS  recovers  the  aircraft,  and  the  chevrons  appear 
approximately  five  seconds  before  the  initiation  of  the  recovery.  In  the  AFTI  project,  the  purpose 
of  the  chevrons  is  primarily  to  provide  precursory  warning  to  the  pilot  before  the  active  GCAS 
takes  control  of  the  aircraft.  Chevron  symbology,  though,  would  also  be  very  useful  if  used  with 
traditional  passive  PGCAS. 

3.1.2  Vertically  Moving  Dispiays 

The  chevron  display  seems  useful,  but  an  ambiguity  exists.  The  horizontal  movement  of 
the  chevrons  does  not  correspond  to  the  vertical  motion  that  the  pilot  must  command  of  the 
aircraft.  Hence,  the  system  lacks  control-display  compatibility.^^  This  could  confuse  the  pilot  in 
times  of  duress  because  the  horizontal  motion  does  not  correspond  to  the  real  world.  The  pilot 
might  forget  whether  the  chevrons’  moving  inward  signals  a  better  or  worse  situation.  One 
possible  solution  is  simply  to  rotate  the  chevrons  so  that  they  move  vertically.  The  bottom 
chevron  moving  upward  in  the  HUD  would  naturally  correspond  to  the  terrain  outside  moving 
upward  in  the  pilot’s  FOV.  The  problem  here  is  that  the  upper  chevron  might  eliminate  the 
correspondence  because  it  would  move  downward  as  the  terrain  came  upward.  The  natural 
solution  is  to  eliminate  the  upper  chevron  (Figure  3.2).  This  would  also  reduce  the  redundancy  of 
two  objects  moving  at  the  same  rate  toward  the  same  location. 


35 


/  \ 

/  \ 


Figure  3.2:  Vertical  Chevron 

If  no  upper  chevron  is  used,  the  pilot  no  longer  sees  an  X  in  the  worst-case  situation.  Also,  the 
chevron  may  be  allowed  to  continue  moving  upward  to  the  top  of  the  HUD  as  the  situation 
deteriorates  beyond  the  point  where  the  horizontal  chevrons  met.  This  change  from  the  standard 
symbology  would  require  additional  training  and  a  paradigm  shift.  The  question  also  arises  of 
what  gain  to  use  for  the  display  and  how  to  limit  the  symbol  to  the  physical  edges  of  the  HUD.  It 
can  only  move  upward  to  the  top  of  the  HUD  glass,  so  further  vertical  motion  might  be  shown  by 
changing  the  shape  of  the  symbol  once  it  is  at  the  top. 

The  vertically  moving  chevron  is  similar  in  principle  to  the  DataBase  Terrain  Cueing 
(DBTC)  display  used  in  the  F-16  Digital  Terrain  System."'^  DBTC  aUows  the  pilot  to  foUow  the 
terrain  at  a  set  clearance  altitude  by  maneuvering  so  that  the  FPM  overlaps  a  box  on  the  HUD. 
The  box  moves  vertically  to  show  the  pilot  how  many  “g”  must  be  commanded  of  the  aircraft  to 
stay  at  the  clearance  altitude.  The  difference  is  that  DBTC  must  be  turned  on  and  off;  it  is  not 
designed  to  alert  the  pilot  of  an  impending  ground  collision.  The  vertical  chevron  is  also  similar 
in  concept  to  the  kinematic  horizon,  which  consists  of  “a  single  line  nominally  across  the  HUD 
indicating  the  horizon  below  which  the  flight  path  vector,  as  depicted  by  the  flight  path  marker, 
must  not  be  allowed  to  go.”*^  The  line  can  even  show  variations  in  vertical  flightpath  available  to 
the  left  and  right  of  the  current  groundtrack,  as  shown  in  Figure  3.3.^^’^^ 
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Figure  3.3:  Kinematic  Horizon 

Both  DBTC  and  the  kinematic  horizon  are  designed  for  terrain  following  (TF).  “For  military 
applications  TF  is  extremely  useful  for  long  range  penetration  missions  but  it  is  far  too  inflexible 
for  Close  Air  Support  and  many  other  roles  requiring  high  maneuverability.”'^  An  advanced 
PGCAS  display  like  the  ones  discussed  here  would  allow  that  maneuverability  and  would  be 
advantageous  in  both  TF  and  non-TF  equipped  aircraft. 

3.1.3  Conformality 

If  a  symbol  in  the  HUD  (the  chevron)  represents  something  outside  the  aircraft  (the 
terrain),  the  question  arises  whether  to  show  the  pilot  the  physical  location  of  the  terrain  by 
making  the  display  conformal.  This  means  superimposing  the  chevron  over  the  terrain  feature 
that  it  represents.  The  question  that  must  be  answered  if  this  type  of  display  is  to  be  used  is 
which  terrain  to  show  the  pilot.  The  display  could  show  the  terrain  elevation  at  a  set  distance 
from  the  aircraft.  The  problem  here  is  that  the  terrain  between  the  aircraft  and  that  point  might  be 
higher  than  the  terrain  displayed  (Figure  3.4). 


Figure  3.4:  Terrain  at  Set  Distance 


Thus,  the  phot  is  not  given  an  accurate  idea  of  the  worst-case  scenario.  A  second  alternative  is  to 
show  the  maximum  terrain  elevation  within  the  scope  of  the  search  polygon.  A  third  option  is  to 
show  the  terrain  that  subtends  the  maximum  elevation  angle  in  the  HUD.  Finally,  one  could 
show  the  worst-case  terrain  as  determined  by  the  PGCAS  algorithm.  Another  question  is  how  to 
show  terrain  that  lies  outside  the  FOV  of  the  HUD.  If  the  most  dangerous  terrain  Ues  just  in  front 
of  the  aircraft,  it  would  subtend  a  large  depression  angle  that  physically  cannot  be  shown  on  the 
HUD.  Other  issues  include  the  symbol  moving  from  one  peak  to  another  and  the  jitter  that  might 
occur.  For  all  of  these  reasons,  it  seems  impractical  to  make  the  vertical  chevron  display 

conformal  even  when  the  benefits  are  considered.  A  better  alternative  is  to  use  a  gain  schedule 
like  that  used  with  the  horizontal  chevrons. 

3.1.4  Multiple  Terrain  Elevations 

The  horizontally  and  vertically  moving  chevrons  provide  rate  information,  making  them 
more  useful  than  the  break-X.  Still,  flight  over  a  steep  ascent  or  a  sheer  valley  could  make  them 
move  quickly  with  no  warning.  If  this  movement  could  be  anticipated,  the  pilot  could  react  more 
smoothly,  hopefully  using  less  effort  and  feeling  more  secure.  To  provide  anticipatory  cues,  one 
solution  is  to  show  predictive  terrain  information.  Showing  the  terrain  at  more  than  one  distance 
would  also  help  the  pilot  determine  why  the  symbol  was  moving,  whether  from  vertical  aircraft 
motion  or  from  changing  terrain  elevation.  Another  benefit  of  showing  more  than  one  terrain 
elevation  is  that  many  of  the  conformality  issues  are  eliminated.  The  two-dimensional  terrain 
profile  no  longer  must  be  compressed  to  one  symbol,  so  the  question  of  which  location  to  show 
no  longer  exists.  The  peak-to-peak  jumps  do  not  occur,  and  the  pilot  can  determine  from 
perspective  and  knowledge  of  the  algorithm  what  the  distance  is  to  various  terrain  features.  The 
display  can  thus  benefit  from  the  well-established  advantages  of  conformality.  Conformal 
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symbology  (representations  of  real-world  features  that  conform  to  environmental  contours)  has 
been  shown  to  enhance  situational  awareness.  The  pilot  uses  his  own  judgment  during  the 
recovery  because  he  can  see  changes  in  the  terrain  height  ahead  of  the  aircraft.  A  conformal 
display  that  shows  multiple  terrain  elevations  is  similar  to  the  synthetic  terrain  display  being 
investigated  for  helmet-mounted  units.  It  is  also  similar  to  the  highway-in-the-sky  being 
evaluated  for  use  in  recovery  and  approach  displays.  ‘The  highway  symbols  under  initial 
evaluation  are  a  series  of  ground-stabilised  bars  with  upward  pointing  ends.  The  pilot  flies  along 
this  highway  without  going  below  the  bars.”^^  The  key  difference  is  that  the  advanced  ground 
collision  avoidance  display  is  used  only  when  danger  is  present.  The  symbology  appears  when 
commanded  by  the  algorithm  and  disappears  when  the  aircraft  is  out  of  danger  or  the  pilot 
acknowledges  the  call.  Synthetic  terrain  research,  though,  can  aid  in  the  development  of  new 
ground  collision  avoidance  displays  to  combine  the  benefits  of  both. 

3.1.5  Perspective  Displays 

The  perspective  display  is  a  natural  way  of  showing  multiple  terrain  elevations  and  could 
be  created  by  extending  the  vertically  moving  display  into  two  dimensions.  The  symbols 
representing  more  distant  terrain  are  made  smaller  to  make  them  appear  farther  away.  One  could 
use  a  series  of  horizontal  lines  or  connect  these  lines  to  form  a  surface,  a  channel,  or  a  tunnel 
(Figure  3.5). 


Figure  3.5:  Perspective  Display  Options 


One  perspective  display  is  the  Spatial  Situation  Indicator  presented  by  Williams  and  Mitchell  for 
head-down  approach  guidance  in  commercial  aircraft.^'^  It  utilized  roll-stabilized  vertical 
“whiskers”  positioned  at  15-second  intervals  out  to  75  seconds  along  the  aircraft  flight  path 
(Figure  3.6).  The  green  lower  portion  extended  from  the  predicted  aircraft  altitude  to  the  terrain, 
showing  predicted  clearance  at  that  point.  The  2000-ft  yellow  portion  was  provided  for 
comparative  reference. 


Predicted 
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Figure  3.6:  Spatial  Situation  Indicator 

The  perspective  information  given  by  this  type  of  display  is  quite  valuable.  The  human  eye  is 
already  calibrated  to  judge  real  distances  and  angles.  “Experiments  with  airline  pilots  showed 
that  the  perspective  symbology  reduced  the  pilot’s  workload  and  at  the  same  time,  improved  the 
tracking  of  the  demanded  flight  path.”^®  The  perspective  display  also  helps  integrate  the  multiple 
sources  of  information  that  the  pilot  would  otherwise  have  to  process:  altitude,  flight  path,  terrain 
altitude,  terrain  slope,  etc.^®  In  an  examination  of  situational  awareness,  Wickens  concludes  that 
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“3D  tunnel  in  the  sky  displays,  in  which  the  viewpoint  of  the  display  corresponds  to  the  position 
of  the  pilot ...  are  superior  for  flight  path  control.”^^ 

A  key  choice  when  a  perspective  display  is  utilized  is  whether  to  fix  it  to  the  earth  or  to 
the  aircraft.  When  fixed  to  the  earth,  it  appears  to  move  past  the  aircraft,  providing  velocity 
cueing.  It  also  provides  instantaneous  roll  information  because  it  moves  with  the  horizon.  These 
cues  are  desirable  because  the  pilot  can  be  more  fully  immersed  and  has  fewer  sources  of 
information  to  assimilate.  When  the  display  is  fixed  to  the  aircraft,  it  can  show  the  terrain  along 
the  flight  path  vector  at  any  time.  This  is  very  helpful  because  the  display  “picks  up”  the  most 
significant  terrain  as  the  aircraft  changes  heading.  The  ideal  PGrCAS  display  functions  equally 
well  whether  the  pilot  decides  to  turn  or  recovers  straight  ahead.  To  provide  guidance  in  any 
direction,  an  earth  fixed  display  must  show  additional  terrain  off  the  flight  path.  This  requires 
much  more  processing  power  and  could  clutter  the  HUD.  To  compromise,  one  can  use  an  aircraft 
fixed  display  that  is  roll-stabilized.  Thus,  the  pilot  gains  roll  cueing  and  can  see  the  terrain  along 
his  flight  path  vector  without  excessive  clutter. 

3.2  Hypothesis 

This  thesis  hypothesized  that  the  more  informative  displays  discussed  above  would 
produce  more  accurate  recoveries  than  the  Break-X.  In  particular,  the  displays  were  hypothesized 
to  be  more  accurate  for  altitude  tracking  after  the  initial  recovery.  The  rate  information  provided 
by  all  but  the  Break-X  would  enable  the  pilot  to  fly  more  smoothly  with  less  effort.  It  was  also 
thought  that  the  pilot  would  feel  more  secure  and  comfortable  using  the  displays  that  provide 
more  information  about  the  terrain  underneath  the  aircraft.  This  project  proposed  four  candidate 
displays  that  fall  into  the  categories  already  mentioned.  The  first  was  a  break-X  similar  to  the 
one  currently  in  use.  The  second  was  a  set  of  chevrons  that  moved  horizontally  inward  to  form 
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the  break-X,  providing  rate  information.  The  third  display  was  a  mountain  symbol  that  moved 
vertically  upward,  providing  the  same  information  as  the  chevrons  but  in  the  vertical  direction. 
The  fourth  display  was  an  elevated  perspective  highway  attached  to  the  aircraft,  which  showed 
upcoming  terrain. 
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4  Experimental  Setup 


Four  PGCAS  displays  were  tested  using  a  fixed-base  T-38  simulator  with  a  projection 
screen  and  simulated  HUD.  Twelve  subjects  flew  through  a  series  of  PGCAS  situations  using 
each  display,  and  data  was  recorded  for  a  period  of  30  seconds  after  each  alert.  They  were 
instructed  to  fly  at  a  set  AGL  altitude  after  receiving  the  alert,  where  they  would  avoid  the  terrain 
but  also  avoid  ground  radar  threats.  Examining  how  well  the  subjects  recovered  to  maintain  the 
set  altitude  allowed  comparison  of  the  displays’  effectiveness. 

4.1  Simulated  PGCAS 

In  order  to  test  the  symbologies,  a  ground  collision  avoidance  algorithm  was  needed  to 
evaluate  the  terrain  and  provide  input  to  the  HUD.  The  algorithm  used  was  a  simplified  version 
of  the  one  implemented  in  many  modern  aircraft.  The  algorithm  consisted  of  two  pieces:  a  pull- 
up  processor  and  a  terrain  processor.  The  pull-up  processor  created  a  two-dimensional  “worst- 
case”  trajectory,  which  accounted  for  a  two-second  pilot  reaction  time,  a  two-second  roll  to 
wings-level,  a  constant-radius  pull-up,  and  a  linear  60°  climb.  The  maximum  load  factor 
available  was  a  linear  function  of  airspeed.  Typical  worst-case  trajectories  are  depicted  in  Figure 
4.1. 

A- 

Descending  Flight  Level  Flight  Climbing  Flight 

Figure  4.1:  Projected  Recoveries 

The  terrain  processor  compared  the  predicted  aircraft  altitude  to  the  terrain  elevation  at  a  series 
of  points  projected  along  the  aircraft’s  groundspeed  vector.  The  points  were  spaced  500  ft  apart 
and  continued  for  a  distance  corresponding  to  10  seconds  at  the  current  groundspeed  (GS).  The 


43 


500-foot  spacing  provided  coverage  of  major  terrain  features  without  slowing  down  the  frame 
rate  noticeably.  Figure  4.2  shows  the  line  of  terrain  locations  examined. 

GS  Vector 


Figure  4.2;  Terrain  Evaluation  Points 

A  real-world  algorithm  would  evaluate  the  terrain  enclosed  by  a  polygon  ahead  of  the  aircraft  to 
account  for  possible  turns.  For  the  simulation,  though,  a  linear  search  pattern  was  sufficient  to 
provide  a  predictive  warning.  The  conqiarison  between  the  projected  recovery  and  the  terrain 
produced  a  series  of  predicted  terrain  clearances,  shown  in  Figure  4.3. 


44 


Altitude  MSL 


Altitude  AGL 

Minimum  Clearance 

j 

500  1000  1500  2000  2500  Distance  from  Aircraft 


Figure  4.3:  Terrain  Processor  Profile 


To  determine  whether  an  alert  would  be  issued,  the  minimum  clearance  (MC)  was  compared  to 
the  three  clearance  heights  shown  in  Figure  4.4.  These  thresholds  were  the  Warning  Clearance 


Height  (WCH),  the  Target  Clearance  Height  (TCH),  and  the  Safety  Clearance  Height  (SCH). 
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Figure  4.4:  Clearance  Altitudes 

The  WCH  was  the  altitude  at  which  the  alert  would  first  appear,  and  the  pilot  was  told  to  stay 
below  the  WCH  during  the  recovery  to  avoid  ground  radar.  The  TCH  was  the  altitude  at  which 
the  pilot  was  instructed  to  fly  during  the  recovery.  The  SCH  was  the  altitude  which  the  pilot  was 
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instructed  not  to  penetrate  for  safety  reasons.  These  altitudes  were  considered  realistic  for  low- 
level  flight  below  ground  radar  coverage.  The  objective  given  the  pilot  was  to  recover  the  aircraft 
to  safety  and  then  fly  at  the  TCH  for  about  30  seconds.  This  would  simulate  a  situation  where  the 
pilot  receives  a  PGCAS  alert  but  wants  to  remain  within  an  altitude  constraint  during  the 
recovery.  The  alerts  provided  cues  to  aid  the  pilot  in  maintaining  this  altitude. 

4.2  Candidate  Displays 

4.2.1  Head-Up  Display 

The  HUD  was  the  primary  source  of  position  and  attitude  information  for  the  pilot. 
Figure  4.5  shows  the  HUD  as  it  appeared  in  the  experiment.  The  HUD  FOV  was  20°  square, 
which  is  realistic  for  a  modern  fighter. 
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Figure  4.5:  HUD  Configuration 

Airspeed  in  knots  and  barometric  altitude  in  feet  were  displayed  as  circular  dials  containing 
digital  readouts  in  the  upper  left  and  right  corners  respectively.  Roll  information  came  fi-om  the 
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rotation  of  the  flight  path  ladder  and  from  the  roll  indicator  scale  and  marker.  The  roll  marker 
triangle  was  split  into  two  sections  that  would  separate  in  a  sideslip.  This  provided  yaw 
information  although  this  rarely  occurred.  Heading  in  degrees  was  displayed  on  a  moving  tape 
scale  across  the  top  of  the  HUD.  The  aircraft  reference  symbol  “W”  was  fixed  to  the  nose  of  the 
aircraft  to  show  pitch.  Finally,  the  Flight  Path  Marker  (FPM)  showed  flight  path  angle  and  was 
referenced  to  the  flight  path  ladder. 

The  four  PGCAS  displays  tested  in  this  experiment  presented  similar  information  in 
differing  formats.  The  Break-X  was  a  discrete  status  display,  which  provided  gross  terrain 
proximity  information.  The  Chevrons  and  Mountain  displays  were  continuous  depictions  that 
provided  position  and  rate  information  in  a  compensatory  format.  The  Highway  was  a 
continuous  display  that  provided  position,  rate,  and  acceleration  information  in  preview  format. 
Figure  4.6  shows  how  each  display  would  appear  when  the  minimum  clearance  was  equal  to  750 
ft  (WCH),  500  ft  (TCH),  250  ft  (SCH),  and  0  ft. 
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Figure  4.6:  Display  Summary 

The  displays  were  driven  by  the  results  of  the  simulated  PGCAS  algorithm.  The  scene- 
generation  software  performed  the  PGCAS  calculations  and  produced  the  alerts  in  the  HUD.  All 
four  were  similar  in  size,  although  they  occupied  different  locations  on  the  HUD.  Their  sizes 
were  based  on  the  100-mrad  standard  that  is  currently  used  in  the  Fairehild  A- 10. 

4.2.2  Break-X 

The  Break-X  (Figure  4.7)  consisted  of  two  100  mrad  lines  crossing  at  45°  angles  to 

produce  a  square  X.  It  appeared  at  the  center  of  the  HUD  when  MC  <  TCH  and  disappeared 
when  MC  >  TCH. 
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Figure  4.7;  Break-X  Display 

The  Break-X  used  in  this  experiment  provided  slightly  more  information  than  is  given  by  the 
same  alert  in  most  operational  aircraft.  In  the  A- 10,  for  instance,  the  “X”  flashes  for  two  seconds 
and  disappears,  giving  no  indication  when  the  aircraft  has  been  sufficiently  recovered.*^  In  this 
simulation,  the  “X”  symbol  was  present  until  the  aircraft  climbed  above  the  TCH  to  safety.  This 
is  the  minimum  standard  that  will  likely  be  used  when  digital  terrain  technology  is  fully 
integrated.  It  was  also  necessary  as  a  baseline  to  allow  comparison  between  the  four  displays. 

The  Break-X  was  binary;  it  gave  on-off  information  but  no  rate  indication.  Because  it 
was  the  only  binary  display,  it  was  expected  to  produce  poorer  quality  recoveries  than  the  other 
three.  In  truth,  the  break-X  was  not  designed  for  a  task  such  as  the  one  tested  here.  It  was 
engineered  to  get  the  pilot’s  attention,  not  to  provide  guidance  during  the  recovery.  When 
presented  in  combination  with  an  audio  warning,  the  break-X  is  a  well-recognized  symbol  that 
alerts  the  pilot  effectively.  The  reason  for  including  it  in  this  project  was  to  show  how  poorly  it 
might  perform  in  a  worst-case  scenario.  Worst-case  means  zero  visibility  flight  near  the  terrain 
where  a  significant  ground  threat  is  present.  In  this  case,  making  an  overly  conservative  recovery 
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to  some  altitude  weU  above  the  terrain  would  prove  impractical  if  not  deadly.  In  visual 
conditions,  the  Break-X  would  gain  back  much  of  its  usefulness.  If  the  pilot  can  quickly  survey 
the  situation  and  recover  appropriately  upon  receiving  the  alert,  it  is  sufficient.  But  even  when 
the  terrain  is  visible,  it  might  be  useful  to  have  some  guidance  on  how  hard  to  pull  and  when  to 
level  off.  And  because  the  HUD  appears  between  the  pilot  and  the  terrain  ahead,  he  can  see  the 
alert  without  focusing  inside  the  cockpit.  Thus,  the  other  alerts  mentioned  here  would  be  unlikely 
to  hinder  the  pilot  during  a  visual  recovery. 

4.2.3  Chevrons 

The  Chevrons  (Figure  4.8)  appeared  at  the  left  and  right  edges  of  the  HUD  when  MC  < 
WCH.  They  moved  inward  as  the  terrain  grew  closer,  and  the  tips  just  touched  two  20-mrad 
vertical  reference  bars  when  MC  =  TCH.  The  chevrons  formed  an  X  in  the  center  of  the  HUD 
when  MC  <  SCH. 
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Figure  4.8:  Chevrons  Display 

The  chevron  spacing  was  based  on  the  distance  between  the  minimum  projected  altitude  and  the 
terrain.  Thus,  the  speed  at  which  the  chevrons  moved  together  showed  the  terrain-closure  rate. 
The  question  arose  of  choosing  the  gain  that  would  determine  chevron  separation.  In  the  end, 
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only  one  gain  would  enable  the  display  to  meet  the  conditions  specified  in  Figure  4.6.  These 
conditions  were  chosen  to  put  the  displays  on  equal  terms,  allowing  comparison  between  them. 
When  the  chevrons  met  in  the  center  of  the  HUD,  the  pilot  saw  the  familiar  Break-X  that  has 
consistently  been  used.  Because  operational  pilots  are  used  to  seeing  the  Break-X,  this  display 
would  probably  be  adapted  to  fairly  easily. 

4.2.4  Mountain 

The  Mountain  (Figure  4.9)  appeared  and  moved  upward  from  the  bottom  of  the  HUD 
when  MC  <  WCH.  It  was  even  with  the  fixed  aircraft  reference  symbol  (“W”)  when  MC  =  TCH. 
It  stopped  moving  vertically  at  the  top  of  the  HUD  when  MC  <  SCH,  and  split  at  this  point  to 
become  flat  across  the  top.  It  touched  the  outside  edges  of  the  HUD  when  MC  =  0,  and  soon 
after  that  the  aircraft  would  impact  the  terrain. 
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Figure  4.9:  Mountain  Display 

This  display  provided  essentially  the  same  information  as  the  chevrons,  but  it  was  oriented 
vertically.  The  mountain  symbol  moved  up  in  the  field-of-view  as  the  aircraft  descended,  just  as 
the  terrain  would  do  if  the  pilot  could  see  it.  The  symbol  could  only  move  vertically  to  the  top  of 
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the  HUD  glass,  so  further  vertical  motion  was  shown  by  the  mountain  spreading,  as  a  real 
mountain  appears  to  spread  when  rising  in  the  field-of-view. 

4.2.5  Highway 

The  Highway  (Figure  4.10)  was  composed  of  horizontal  and  vertical  lines  drawn  to  show 
a  perspective  view  of  an  elevated,  roll-stabilized  surface  at  the  TCH.  The  legs  of  the  highway 
extended  to  the  terrain.  The  longitudinal  spacing  was  chosen  to  show  each  terrain  location  being 
examined  by  the  PGCAS  algorithm,  so  the  sets  of  legs  were  500  ft  apart.  The  width  of  the 
nearest  bar  just  filled  the  20®  FOV.  The  bars  became  narrower  as  they  showed  more  distant 
terrain.  This  is  similar  to  the  terrain  contour  specified  in  MIL-STD-81641,  which  consists  of 
horizontal  lines  representing  elevation  angles  to  the  highest  terrain  within  each  of  five  range 
mcrements.**  The  main  difference  was  that  the  Highway  had  more  than  five  range  increments  in 
order  show  all  of  the  terrain  being  evaluated  by  the  terrain  processor. 


Figure  4.10:  Highway  Display 

For  the  sake  of  the  experiment,  it  seemed  prudent  to  show  the  actual  location  of  the  TCH  since 
this  was  the  goal  given  the  pilot.  The  location  of  the  actual  terrain  was  also  important  to  show 
since  the  real  danger  existed  at  that  level.  The  elevated  highway  symbology  was  chosen  to  show 
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these  two  altitudes,  and  it  had  the  additional  benefit  of  resembling  a  physical  surface.  The 
Highway  was  the  only  conformal  display  of  the  four.  If  the  pilot  were  able  to  see  the  terrain,  it 
would  have  been  at  or  just  below  the  base  of  the  legs.  The  surface  showed  the  physical  location 
of  the  Target  Clearance  Height,  500  ft  above  the  terrain. 

4.3  Hardware  and  Software 

The  simulation  designed  to  test  the  four  candidate  displays  used  a  fixed-base  T-38 
cockpit  facing  a  6’x8’  projection  screen.  The  synthetic  HUD  was  projected  on  the  screen,  and 
simulated  fog  obscured  the  visual  scene.  The  entire  simulation  setup  resided  on  three  processors 
-  the  first  hosted  the  T-38  vehicle  dynamics  and  timing  routines,  the  second  hosted  the  outside 
scene  and  HUD,  and  the  third  hosted  the  T-38  cockpit  hardware  input  and  output.  Figure  4. 1 1 
shows  a  block  diagram  of  the  integrated  simulation  setup  with  specifications  for  each  processor. 
All  processors  communicated  through  a  common  ethernet  connection.  The  vehicle  dynamics, 
scene  generator,  and  cockpit  FO  programs  broadcasted  and  gathered  information  in  a 
standardized  format  over  a  Network  DataBase  (NDB).  The  simulator  program  sent  and  received 
signals  directly  through  a  UNIX  socket  connection. 
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Figure  4.11:  Simulation  Hardware  Block  Diagram 

4.3.1  T-38  Vehicle  Dynamics 

The  Draper  Laboratory  T-38  vehicle  dynamics  simulator  was  chosen  because  of  its 
robustness,  flexibility,  and  commonality.  Since  it  is  the  preliminary  trainer  in  the  fighter  track,  all 
pilots  have  flown  the  T-38.  The  vehicle  dynamics  program  sent  outputs  to  and 
received  inputs  fi-om  the  scene  and  cockpit  via  the  NDB.  This  program  was  the  core  of  the 
integrated  flight  simulator.  In  addition  to  calculating  the  vehicle  dynamics,  it  managed 
environmental  factors  and  included  data  recording  capabilities.  The  simulation  ran  at  about  120 
Hz  (time-step  of  0.0085  sec). 

4.3.2  T-38  Cockpit 

The  T-38  cockpit  was  used  to  make  the  simulated  flight  experience  more  realistic.  It 
provided  a  pilot  interface  consistent  with  operating  an  actual  fighter.  The  cockpit  included  a 
number  of  analogue  and  discrete  channels  for  controls.  The  controls  utilized  for  this  experiment 
were  the  stick  and  rudder  pedals.  The  trim  and  flap  switches  provided  inputs  to  the  simulation. 
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but  remained  fixed  at  neutral  and  flaps  up  respectively.  The  two  throttles  were  fixed  at  full 
military  power.  The  cockpit  appears  in  Figure  4.12. 


Figure  4.12;  T-38  Cockpit 


4.3.3  Data  Recording 

Variables  recorded  included:  time,  aircraft  position  and  rates,  Euler  angles,  rates,  and 
accelerations,  stick  position,  load  factor,  alert  symbology  position  on  the  HUD,  and  terrain 
altitude.  The  data  were  recorded  at  a  rate  of  12  Hz.  The  simulation  iBramework  included  the 
capability  to  log  data,  which  were  converted  and  analyzed. 
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4.4  Simulation 


4.4.1  Visual  Scene 

The  HUD  was  generated  using  the  Gemini  Visual  System  (GVS)  SIMation  Series 
Software  graphics  package.  The  GVS  software  included  functions  for  building  and  manipulating 
scene  objects  and  creating  viewing  “cameras”  and  attaching  them  to  objects,  as  well  as  pre-built 
objects  and  textures  to  be  used  in  a  scene.  Nothing  in  the  scene  was  visible  to  the  pilot  except  the 
HUD  and  PGCAS  symbology,  drawn  as  black  lines  against  a  cloud-gray  background.  The  frame 
rate  produced  by  the  GVS  software  was  generally  near  30  Hz.  The  GVS  software  was  also  used 

to  perform  the  simulated  PGCAS  calculations  because  of  its  immediate  access  to  the  terrain 
database. 

4.4.2  Flight  Director 

The  desire  was  to  test  the  four  displays  at  a  variety  of  flight  path  and  bank  angles.  To 
achieve  a  specified  alert  attitude,  the  pilots  followed  a  series  of  commands  given  by  a  flight 
director  on  the  HUD.  The  flight  director  symbology  appears  in  Figure  4.13.  The  pilots  pitched  to 
superimpose  the  FPM  over  a  flight  director  dot  and  banked  to  match  the  wings  of  the  FPM  with  a 

Tracking: 

Flight  Path  Marker  properly 
aligned  with  flight  director 


Flight  Director  Symbology 


line  across  the  dot. 

Command: 

Pitch  down  and  left 
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The  flight  director  would  remain  at  a  given  flight  path  and  bank  angle  until  the  pilot  was  able  to 
match  the  command  within  ±2°  of  flight  path  and  +5°  of  bank  for  4  seconds.  It  would  then 
smoothly  transition  to  the  next  position  on  the  HUD.  The  subject  followed  the  flight  director 
through  one,  two,  or  three  conunands  before  reaching  the  final  alert  attitude. 

4.4.3  Terrain 

The  terrain  database  was  created  using  satellite  imagery  taken  in  Yellowstone  Park, 
Wyoming.  The  image  shown  in  Figure  4.14  was  texture-mapped  over  multiple  polygons  created 
from  digital  terrain  data,  creating  three  dimensional  mountains,  valleys,  and  plains.  The  terrain 
block  represented  in  the  photograph  is  7.5  NM  square,  and  the  picture  is  oriented  with  North  up. 


Figure  4.14:  YeUowstone  Park  Terrain 
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This  terrain  block  was  mirrored  multiple  times  to  create  a  database  large  enough  so  the  pilots 
could  not  fly  off  the  edge  of  the  terrain.  Since  the  terrain  was  invisible  in  the  simulation,  only 
terrain  elevation  information  was  taken  from  this  database.  The  PGCAS  algorithm  could  query 
the  database  for  terrain  elevation  at  any  X-Y  location.  Linear  interpolation  enabled  the  algorithm 
to  determine  the  terrain  elevation  between  two  data  posts. 

It  was  desirable  for  the  pilots  to  receive  each  alert  at  the  same  x-y  location,  altitude,  and 
headmg.  This  would  make  the  terrain  encountered  nearly  identical  and  would  aid  in  comparison 
of  the  resulting  trajectories.  To  mechanize  this,  the  simulated  aircraft  was  programmed  to 
instantly  jump  to  a  pre-specified  location  and  heading  when  the  pilot  met  the  final  command. 
This  was  not  apparent  to  the  pUot  because  the  altitude  and  heading  jumps  were  not  shown  on  the 
HUD  and  the  terrain  was  invisible.  This  was  important  to  ensure  the  pilot  was  not  aware  when 
the  alert  would  be  triggered.  The  aircraft  always  jumped  to  an  altitude  such  that  the  PGCAS 
algorithm  would  trigger  an  alert  approximately  one  second  after  the  jump.  The  aircraft  jumped  to 
one  of  two  different  locations  where  it  would  encounter  either  smooth  or  rugged  terrain.  The 
purpose  for  using  the  two  terrain  types  was  to  determine  whether  some  displays  performed  better 
than  others  did  over  a  certain  type  of  terrain.  Figure  4.15  shows  representative  terrain  profiles  for 
the  smooth  and  rugged  terrain.  All  of  the  profiles  encountered  had  the  same  basic  shape  as  these 
two,  any  variations  were  caused  by  differing  bank  angles  between  successive  runs. 
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Smooth  Terrain  Profile 


Rugged  Terrain  Profile 


Figure  4.15:  Terrain  Encountered 

For  the  purposes  of  the  simulation,  the  aircraft  was  allowed  to  fly  lower  than  the  terrain  surface 
without  pausing  the  simulation.  This  was  artificial  since  a  crash  would  occur  in  the  real  world, 
but  it  was  necessary  to  provide  continuous  altitude  information.  To  inform  the  pilot  when  the 
aircraft  was  in  the  crash  condition  (AGL  altitude  less  than  zero),  a  black  shape  resembling  a 
bent-up  pitot  tube  was  shown  on  the  HUD.  This  happened  very  seldom,  and  any  pilots  who  saw 
the  pitot  tube  were  already  in  the  midst  of  a  recovery.  Thus,  the  crash  indication  had  very  little 
effect  on  the  pilot  actions. 
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4.5  Experimental  Design 

4.5.1  Protocol 


Each  subject  was  told  the  purpose  of  the  experiment  and  given  some  background  on 
PGCAS.  He  then  read  the  directions  (Appendix  A)  and  signed  the  consent  form  (Appendix  B). 
The  subject  became  familiar  with  the  controls  and  was  given  time  to  practice  following  the  flight 
director.  Next,  he  was  allowed  to  practice  with  the  first  display  and  ask  questions  about  it.  Once 
comfortable  using  the  display,  the  subject  flew  six  runs.  Each  run  consisted  of  following  a  series 
of  HUD  visual  commands  to  place  the  aircraft  in  a  predetermined  alert  attitude.  A  PGCAS  alert 


was  triggered,  the  pilot  recovered,  and  data  were  recorded  for  about  30  seconds.  The  six  runs 
were  followed  by  an  oral  interview  before  moving  on  to  the  next  display.  The  experimenter 
recorded  the  pilot’s  responses  to  the  survey  questions,  included  in  Appendix  C.  The  6  runs  and 
mterview  were  repeated  for  each  of  the  other  three  displays.  Then  the  subject  exited  the  cockpit 
and  filled  out  the  remainder  of  the  questionnaire.  Each  subject  flew  6  runs  on  each  of  4  displays 
for  a  total  of  24  runs.  The  set  of  6  runs,  which  appears  in  Table  4.1,  was  identical  between 
displays  and  between  subjects.  The  table  Usts  the  flight  path  and  roll  angles  in  degrees  along  with 
the  type  of  terrain  encountered  during  that  run. 


Table  4.1:  Test  Matrix 


Run 


Terrain 


1 

2 

3 

4 
5_ 
6 


Smooth 

Rugged 

Smooth 

Rugged 

Smooth 

Rugged 


(15,-15) 
(-5, 15) 
(-25,  30) 
(15,-15) 
(-5, 15) 
(-25,  30) 
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4.5.2  Counterbalancing 

The  experiment  was  counterbalanced  to  account  for  learning,  fatigue,  and  other  order 
effects;  Table  4.2  shows  the  resulting  order 


Table  4.2:  Balanced  Design 
Subject 


1®‘  Display 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

X 

X 

H 

H 

H 

H 

X 

X 

C 

M 

C 

M 

2"'*  Display 

C 

M 

C 

M 

X 

X 

H 

H 

M 

C 

M 

C 

3'^^  Display 

M 

C 

M 

C 

C 

M 

C 

M 

X 

X 

H 

H 

4®  Display 

H 

H 

X 

X 

M 

C 

M 

C 

H 

H 

X 

X 

X  =  Break-X  C  =  Chevrons  M  =  Mountain  H  =  Highway 

To  help  reduce  the  number  of  required  subjects,  the  displays  were  balanced  in  a  blocked  fashion. 

The  Break-X  was  iconic  and  discrete,  standing  by  itself.  The  Chevrons  and  Mountain  presented 
similar  continuous  information  in  horizontal  and  vertical  formats,  so  they  were  blocked  together. 
The  Highway  presented  predictive  information  and  also  stood  by  itself. 
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5  Results 


5.1  Subject  Demographics 

The  subject  pool  consisted  of  military  and  civilian  graduate  students  and  employees  of 
Draper  Laboratory.  The  subjects  were  required  to  have  a  Private  Pilot  rating  or  equivalent,  which 
would  enable  them  to  understand  the  simulation.  This  ensured  that  the  subjects  could  correlate 
their  experience  with  real-world  flying  and  that  they  were  familiar  with  aircraft  controls.  The 
twelve  subjects,  all  male,  had  an  average  of  820  total  flight  hours.  Six  had  experience  in 
instrument  conditions,  nine  had  simulated  or  real  experience  with  Head-Up  Displays,  and  ten  had 
military  flight  experience.  The  subjects  ranged  from  22  to  54  years  old  with  a  mean  age  of  3 1. 

5.2  Subject  Performance 

To  compare  the  four  displays  anal34ically,  an  Analysis  of  Variance  (ANOVA)  was 
applied  to  the  data  to  obtain  the  F-statistic  with  95%  confidence  (a  =  0.05).  This  would 
determine  whether  performance  differences  were  due  to  chance  or  to  the  differing  alerts.  If  there 
was  a  significant  difference  between  the  displays,  the  Link-Wallace  test  was  applied  to  determine 
the  critical  distance  (dc)  between  means^°.  The  distance  dc  is  the  minimum  difference  between 
means  that  must  exist  in  order  to  be  statistically  significant.  The  critical  distance  was  compared 
to  the  mean  difference  between  each  pair  of  displays  to  determine  which  ones  were  more 
effective  than  others.  The  error  bars  shown  on  the  following  plots  represent  dc,  so  a  significant 
difference  exists  if  the  bar  height  for  one  display  exceeds  the  error  bar  for  another. 

5.2.1  Roll  to  Wings-level 

The  first  part  of  the  recovery  involved  the  roll  to  wings-level.  One  of  the  items  being 
investigated  was  whether  any  of  the  displays  performed  significantly  better  in  helping  the  pilot 
roll  initially  and  to  maintain  the  wings-level  attitude  throughout  the  recovery.  Leveling  the 
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aircraft  quickly  is  important  because  the  minimum  altitude  is  lost  during  the  recovery  if  the  roll 
occurs  before  the  pull-up.  With  some  of  the  displays,  the  pilots  tended  to  recover  the  aircraft 
while  maintaining  a  bank  angle,  which  could  prove  dangerous  in  a  real  situation.  Figure  5.1 
shows  time  histories  of  roll  angle,  which  represents  roll  error  since  the  desired  angle  was  zero. 
The  alert  was  issued  at  time  zero,  and  the  grouping  during  the  first  few  seconds  resulted  from  the 
three  initial  roll  angles:  30°,  -30",  and  0°. 


Break  X 


0  10  20  30 
Time  (sec) 


Mountain 


Time  (sec) 


Chevrons 


Highway 


Figure  5.1:  RoU  Angle  Histories  (72  cases) 

The  figure  shows  that  the  Break-X  produced  numerous  recoveries  at  significant  roll  angles.  The 
Highway  produced  the  tightest  grouping,  followed  by  the  Chevrons.  Note  how  quickly  the 
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Highway  grouping  narrows  to  near  zero.  This  reinforces  pilot  comments  that  the  Highway 
naturally  cued  the  pilot  to  roll  to  wings-level. 

Because  roll  in  either  direction  is  equally  undesirable,  the  metric  used  for  roll 
performance  was  RMS  roll  angle,  averaged  between  runs  and  between  subjects.  Figure  5.2 
shows  the  RMS  roll  angle  for  the  four  displays. 


Break-X  Chevrons  Mountain  Highway 

Figure  5.2:  RMS  Roll  Angle  (n  =  72) 

One  can  see  that  the  Break-X  produced  significantly  more  roll  error  than  the  other  three  displays. 
This  is  somewhat  surprising  because  with  the  binary  alert,  the  pilot  did  not  have  a  tracking  task 
like  he  did  with  the  other  three  displays.  One  might  expect  that  the  pilot  could  pay  more  attention 
to  bank  angle  without  the  tracking  task  and  therefore  accumulate  less  RMS  roll  error.  One  reason 


65 


for  the  poorer  Break-X  performance  may  have  been  the  sense  of  urgency  the  symbol  implied. 
When  the  pilot  saw  the  X,  he  knew  that  danger  was  already  very  present  and  tended  to  pull  back 

immediately.  The  other  displays  “eased”  the  pilot  into  the  warning  situation,  enabling  him  to  roll 
first  and  then  pull. 

Of  the  four  displays,  the  Highway  was  the  only  roll-stabilized  symbology.  That  is,  the 
symbology  rolled  to  conform  with  the  horizon  instead  of  with  the  aircraft.  The  other  three  were 
fixed-axis  and  required  the  pilot  to  keep  track  of  roll  separately  from  pitch  via  the  roll  indicator. 
Previous  work  related  to  attentional  effects  with  superimposed  symbology  suggests  that  “the 
efficient  processing  of  two  information  sources  is  only  possible  when  they  are  part  of  the  same 
perceptual  object.”^  ^  Since  it  rotated  with  the  horizon,  the  Highway  was  a  single  perceptual 
object  that  incorporated  both  bank  and  pitch  information.  This  may  have  allowed  roll  angle  and 
altitude  to  be  processed  more  efficiently.  One  can  see  firom  the  chart  that  the  Highway  did  indeed 
produce  the  least  roll  error,  although  it  was  not  significantly  different  firom  the  Chevrons  or 
Mountain.  Also,  the  Mountain  performed  slightly  worse  than  the  Chevrons.  This  coincides  with 
comments  fi'om  eight  subjects  who  said  the  mountain  symbol  moving  vertically  gave  the  illusion 
that  the  aircraft  was  wings-level,  when  in  fact  it  may  have  had  significant  bank.  A  better 
symbology  would  roll  stabilize  either  the  mountain  icon  or  the  y-axis  along  which  it  moves.  One 
or  both  of  these  would  likely  improve  the  roll  performance  of  this  display.  In  the  AFTI  F-16,  the 
HUD  chevrons  are  roll-stabilized,  and  this  should  probably  be  incorporated  in  any  chevron-type 
display. 

5.2.2  Altitude  Maintenance 

The  main  concern  in  comparing  the  four  alerts  was  how  much  they  helped  with 
maintaining  the  target  altitude.  The  pilot  could  not  see  the  terrain  during  the  recovery,  and  the 
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HUD  altimeter  showed  barometric  altitude.  Thus,  the  PGCAS  display  provided  the  sole  cue  of 
AGL  altitude.  The  pilots  were  instructed  to  maintain  the  target  clearance  height  (500  ft  AGL). 
Thus,  altitude  error  can  be  defined  as  AGL  Altitude  -  TCH.  Time  histories  of  altitude  error  for 
the  four  displays  appear  in  Figure  5.3. 
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Figure  5.3:  Altitude  Error  Histories  (72  cases) 

The  figure  shows  progressively  tighter  grouping  as  one  moves  from  Break-X  to  the  Highway. 
This  shows  that  pilots  were  better  able  to  maintain  the  target  altitude  using  the  dynamic  displays. 
Many  of  the  Break-X  trajectories  show  error  up  to  2000  ft,  while  the  Chevrons  and  Mountain 
plots  show  only  a  few  runs  where  this  much  error  was  reached.  The  Highway  shows  the  least 
error,  and  only  two  trajectories  lie  outside  the  main  group  firom  -500  to  +500  ft. 
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To  better  quantify  the  altitude  performance  from  the  four  displays,  one  can  examine  the 
fraction  of  total  time  accumulated  at  each  altitude.  The  four  histograms  shown  in  Figure  5.4  are 
cumulative  over  all  of  the  pilots  and  over  all  of  the  terrain  and  entry  conditions.  Two  vertical 
dashed  lines,  provided  for  reference  on  each  plot,  show  the  Safety  Clearance  Height  and 
Warning  Clearance  Height  at  250  ft  and  750  ft  respectively.  The  mean  altitudes  for  the  four 
displays  are  also  shown.  Negative  altitudes  have  been  lumped  to  produce  the  tall  bar  at  zero, 
which  represents  the  fraction  of  time  the  pilots  spent  in  the  crash  condition.  Altitudes  above  2000 
ft  have  also  been  lumped  to  produce  the  tall  bar  at  2000.  The  figure  represents  probability 
density  for  altitude,  so  the  area  under  each  curve  is  unity.  Because  the  target  altitude  was  500  ft 
AGL,  the  ideal  curve  would  have  a  tall  spike  at  500  with  little  area  in  the  tails. 


Figure  5.4:  AGL  Altitude  Histogram  (n  =  72) 
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The  curves  are  skewed  to  the  right,  toward  higher  altitudes.  This  is  probably  because  pilots 
wanted  to  err  to  the  high  side,  especially  with  the  Break-X  because  it  provided  so  little 
information.  The  Break-X  produces  a  widely  spread  distribution  with  large  area  in  the  tails.  The 
plot  shows  significant  probability  at  and  above  2000  ft  AGL,  well  above  the  altitude  where  a 
SAM  shootdown  becomes  likely.  The  fraction  of  time  spent  near  the  target  altitude  is  quite  small 
since  the  curve  peaks  at  just  over  2.5%.  The  Chevrons  and  Mountain  produce  better 
distributions,  since  the  curves  peak  near  8%  at  the  target  altitude.  The  Mountain  curve  is  taller 
and  narrower  than  the  Chevrons,  denoting  better  performance.  The  Highway  produces  the  most 
ideal  histogram  with  a  maximum  near  15%  and  very  little  area  in  the  tails.  The  amount  of  time 
spent  outside  the  desired  range  of  250-750  ft  is  smaller  than  for  any  of  the  other  displays.  This 
shows  that  the  Highway  allowed  the  pilots  to  spend  the  most  time  near  the  target  altitude. 

The  altitude  range  shown  on  the  histogram  can  be  divided  into  four  bins:  below  the 
terrain,  between  the  terrain  and  SCH,  between  the  SCH  and  WCH,  and  above  the  WCH.  Figure 
5.5  shows  the  cumulative  fi’action  of  total  time  for  each  category.  Crashed  includes  all  negative 
altitudes  and  is  the  least  desirable  category.  Negative  altitudes  were  possible  because  the 
simulation  continued  to  run  even  when  the  aircraft  was  below  the  terrain.  This  allowed  multiple 
crashes  during  a  single  run  for  better  display  comparison.  Next  comes  Terrain-to-SCH,  which 
represents  altitudes  from  0  to  250  ft  AGL.  This  category  was  considered  dangerous  but  not  fatal 
in  the  simulation.  SCH-to-WCH  is  the  altitude  range  in  which  the  pilots  were  instructed  to  fly, 
250  to  750  ft  AGL.  Finally,  the  area  Above-WCH  shows  the  fraction  of  time  pilots  exceeded  the 
WCH  (750  ft  AGL).  Time  spent  in  this  category  meant  risk  of  being  targeted  by  ground  threats. 
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Figure  5.5:  Time  Spent  in  Altitude  Categories  (n  =  72) 

The  Crashed  category  decreases  steadily  from  Break-X  to  Highway,  although  it  is  relatively 
small  for  all  four  displays.  The  section  representing  Terrain-to-SCH  also  becomes  smaller  across 
the  four  displays,  meaning  that  pilots  spent  less  time  in  the  dangerous  region  below  the  Safety 
Clearance  Height.  The  area  showing  SCH-to-WCH  grows  steadily,  which  is  consistent  with  the 
histogram  analysis  above  and  means  that  pilots  spent  more  time  within  the  accepted  altitudes. 
Smce  the  above-WCH  category  lessens  across  the  displays,  pilots  tended  to  exceed  the  WCH 
less  often  using  the  more  informative  displays.  This  allowed  them  to  remain  in  the  acceptable 
altitude  range  more  consistently  by  using  the  display  guidance  to  level  off. 
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The  Crashed  category  is  very  important  since  this  is  the  most  catastrophic  of  all  possible 


outcomes.  Thus,  it  has  been  expanded  in  Figure  5.6  to  allow  better  comparison. 


Break-X  Chevrons  Mountain  Highway 


Figure  5.6:  Time  Spent  in  Crash  Condition  (n  =  72) 

A  significant  difference  exists  between  all  of  the  displays  except  between  the  Chevrons  and 
Mountain.  The  Chevrons  and  Mountain  presented  similar  information,  so  it  is  expected  that  they 
would  produce  comparable  results.  The  Mountain  gave  additional  guidance  as  the  aircraft 
descended  from  the  SCH  to  the  terrain  by  spreading  across  the  top  of  the  screen,  while  the 
Chevrons  formed  an  X  at  the  SCH  and  remained  that  way  until  a  collision.  This  may  have 
contributed  to  the  difference  that  does  exist,  although  it  is  not  statistically  significant.  The 


71 


Highway  produced  the  least  time  in  the  crash  condition,  probably  because  the  pilots  were  more 
aware  of  the  terrain  ahead  and  could  implement  control  inputs  earlier. 

Another  metric  which  can  be  used  to  determine  the  effectiveness  of  the  displays  in 
preventing  crashes  is  the  discrete  number  of  crashes,  defined  by  the  aircraft’s  altitude  going  fi-om 
positive  to  negative.  This  is  slightly  different  than  the  time  spent  below  the  terrain  because  a 
pilot  who  was  more  aware  of  his  altitude  should  have  recovered  earlier  and  spent  less  time  below 
the  terrain  during  each  crash.  Figure  5.7  compares  the  average  number  of  crashes  per  run. 


Figure  5.7:  Number  of  Crashes  per  Run  (n  =  72) 

Because  these  data  were  produced  by  counting  crashes,  the  F-test  for  two  counts  based  on  the 
Poisson  distribution  applies.^®  This  test  does  not  determine  a  critical  mean  difference,  but  ean 
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determine  whether  a  significant  difference  exists  between  each  pair  of  displays.  All  of  the 
differences  are  significant  except  for  the  one  between  the  Chevrons  and  Mountain  (95% 
confidence).  Again,  the  Highway  outperformed  the  Chevrons  and  Mountain  which  in  turn  did 
better  than  the  Break-X.  Note  the  similarity  between  Figure  5.6  and  Figure  5.7.  Since  both  plots 
show  the  same  relative  bar  heights  and  trends,  it  is  reasonable  to  infer  that  both  “time  crashed” 
and  “number  of  crashes”  are  similar  indicators  of  display  performance. 

Another  item  of  great  concern  in  this  project  was  to  determine  how  well  the  various 
displays  allowed  the  pilot  to  level  off  after  the  initial  pull-up.  This  would  not  apply  to  civilian 
aircraft  because  they  generally  do  not  have  a  tactical  ceiling.  For  military  flying,  though,  it  could 
mean  the  difference  between  preserving  and  losing  the  stealth  which  terrain  masking  provides. 
One  way  to  compare  level-off  performance  is  to  see  how  many  times  the  pilots  penetrated  the 
WCH,  below  which  they  were  instructed  to  stay.  Figure  5.8  compares  the  number  of  WCH  busts, 
and  all  of  the  differences  are  significant  except  for  between  the  Chevrons  and  Mountain. 
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Figure  5.8:  Number  of  Altitude  Busts  per  Run  (n  =  72) 

The  Highway  produced  the  least  ceiling  penetrations,  followed  by  the  Chevrons  and  Mountain. 
The  Break-X  was  significantly  worse  in  this  area  than  the  other  three  alerts;  the  average  was 
more  than  one  penetration  during  each  run.  The  primary  reason  was  probably  the  binary  nature 
of  the  Break-X.  When  it  disappeared,  the  pilots  theoretically  should  have  leveled  off.  They  often 
continued  to  climb,  though,  because  they  wanted  to  be  sure  the  “X”  did  not  reappear.  The  tension 
between  penetrating  the  ceiling  and  receiving  another  alert  was  noticeable,  and  often  the  pilots 
chose  to  penetrate  the  WCH  rather  than  receive  another  Break-X.  The  other  alerts  provided 
feedback  as  the  pilots  climbed,  and  they  were  able  to  level  off  quickly  at  the  TCH  because  the 
symbology  remained  on  the  HUD. 
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5.2.3  Altitude  Trends 


There  are  several  ways  to  quantify  altitude  error  for  analysis.  One  method  simply 
averages  the  errors  at  all  of  the  timesteps  to  produce  a  mean  altitude  error  for  each  run.  These 
errors  were  averaged  between  runs  and  subjects  to  produce  Figure  5.9.  One  must  remember  that 
a  positive  error  at  one  timestep  tends  to  cancel  a  negative  error  at  another,  reducing  the  overall 
sum.  Thus,  averaging  is  most  useful  to  see  whether  the  pilots  tended  high  or  low. 


600 


Break-X  Chevrons  Mountain  Highway 


Figure  5.9:  Mean  Altitude  Error  (n  =  72) 

The  figure  shows  that  the  pilots  tended  to  fly  progressively  lower  (closer  to  the  target  altitude)  as 
one  moves  across  the  displays  from  left  to  right.  The  pilots  intentionally  or  inadvertently  tended 
to  err  high  with  the  Break-X.  This  probably  means  they  flew  more  conservatively  due  to  the  lack 
of  information  present.  They  were  more  confident  with  the  more  informative  dynamic  displays 
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since  they  provided  more  knowledge  of  the  terrain.  This  allowed  the  subjects  to  fly  lower 
because  they  had  less  fear  of  CFTT.  The  Highway  produced  the  lowest  trends,  and  the  pilots  felt 
the  most  confident  using  it. 

The  magnitudes  of  the  averaged  errors  are  not  necessarily  comparable  because  positive 
and  negative  errors  cancel  one  another.  The  magnitudes  can  be  compared,  though,  by  averaging 
the  absolute  value  of  error  instead  of  the  error  itself.  The  resulting  data  is  shown  in  Figure  5. 10. 
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Figure  5.10:  Mean  Magnitude  of  Altitude  Error  (n  =  72) 

When  the  absolute  value  of  the  error  is  used  as  a  metric,  the  displays  again  produce  progressively 
less  error  across  the  four  displays.  The  error  bars  show  the  least  significant  difference,  so  the 
other  three  displays  performed  significantly  better  than  the  Break-X.  The  Highway  produced 
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significantly  less  error  than  the  other  displays,  and  the  Mountain  was  better  than  the  Chevrons 
although  the  difference  is  not  significant.  The  dashed  line  shows  the  location  of  the  Warning 
Clearance  Height  and  the  Safety  Clearance  Height,  respectively  250  ft  above  and  below  the 
TCH.  These  were  the  limits  within  which  the  subjects  were  instructed  to  stay.  One  can  see  that 
the  Break-X  error  is  well  outside  the  bound,  meaning  that  the  subjects  as  a  group  did  not  achieve 
the  goal  stated.  The  Chevron  error  lies  just  inside  the  bound,  while  the  Mountain  and  Highway 
errors  are  well  below  the  line.  This  means  that  all  three  dynamic  displays  enabled  the  pilots  on 
average  to  achieve  the  altitude  following  goal. 

5.2.4  Terrain  Effects 

Runs  using  the  two  different  terrain  types  can  be  separated  as  in  Figure  5.11.  This  allows 
comparison  of  the  four  displays  over  smooth  and  rugged  terrain.  The  performance  metric  shown 
on  this  chart  is  RMS  altitude  error,  which  is  similar  to  the  mean  error  magnitude.  The  difference 
is  that  the  square  root  was  taken  after  dividing  by  the  number  of  cases.  This  means  large  errors 
were  weighted  more  heavily  than  small  errors,  and  this  is  appropriate  since  danger  increases 
rapidly  with  altitude  error. 
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Figure  5.11:  Terrain  Effects  on  Altitude  Error  (n  =  72) 

The  figure  shows  that  the  three  continuous  displays  produce  similar  performance  over  smooth 
terrain;  all  of  them  perform  better  than  the  Break-X.  Over  rugged  terrain,  though,  a  significant 
difference  exists  between  the  three  displays.  The  Highway  is  significantly  better  than  the 
Mountain,  which  is  in  turn  better  than  the  Chevrons.  This  is  not  surprising,  because  following 
rugged  terrain  demands  more  control  input  from  the  pilot.  If  the  terrain  had  been  completely  flat, 
the  pilot  could  theoretically  have  noted  the  altitude  where  the  Break-X  appears  and  leveled  off 
just  above  it  without  even  using  the  symbology.  But  over  rugged  terrain,  a  display  must  provide 
more  active  guidance  for  accurate  terrain  masking.  This  guidance  was  present  in  the  dynamic 
displays.  The  Highway  performed  significantly  better  than  the  other  displays  over  rugged  terrain. 
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The  predictive  surface  enabled  pilots  to  evaluate  terrain  features  ahead  of  the  aircraft  before 
reaching  them.  Thus,  their  corrections  were  immediate  and  appropriate  instead  of  lagging  behind 
the  symbology. 

5.2.5  Dive  Angle  Effects 

When  the  alerts  occurred,  pilots  were  near  one  of  three  dive  angles:  5°,  15",  or  25°.  The 
purpose  for  testing  differing  flight  path  angles  was  to  see  whether  steeper  angles  would  produce 
more  altitude  error.  Figure  5.12  shows  the  results  with  smooth  and  rugged  terrain  runs  separated. 


Figure  5.12:  Initial  Dive  Angle  Effects  on  Altitude  Error  (n  =  72) 

The  figure  shows  that  a  25°  dive  angle  produced  more  altitude  error  than  5°  or  15°.  Aircraft 
diving  more  steeply  took  longer  during  the  initial  recovery  to  stabilize  near  the  TCH,  which  may 
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have  contributed  to  the  larger  error.  More  oscillation  about  the  TCH  was  expected  for  steeper 
entries  because  pilots  would  tend  to  overshoot  the  TCH  initially,  especially  using  the  Break-X. 
The  other  three  displays  provided  some  advance  warning  so  the  pilot  could  anticipate  the 
recovery  and  time  it  to  achieve  the  TCH  more  quickly.  Initial  roll  angle  was  not  found  to  have 
any  significant  effects  on  altitude  performance. 

5.2.6  Age  Effects 

It  was  wondered  whether  the  age  of  the  subjects  would  have  any  effect  on  performance. 
This  was  not  just  for  curiosity,  but  also  related  to  how  well  pilots  might  adapt  to  a  new  PGCAS 

display.  Figure  5.13  is  a  plot  of  RMS  altitude  error  versus  age,  including  a  best-fit  linear 
regression. 


Figure  5.13:  Age  Effects  on  Altitude  Error  (n  =  12) 
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One  can  see  that  age  and  performance  in  this  experiment  were  correlated.  A  t-test  for  correlation 
determined  with  95%  confidence  that  the  correlation  was  significant.  There  are  several  reasons 
why  older  subjects  may  have  followed  the  TCH  less  accurately.  First,  reflexes  and  hand-eye 
coordination  are  known  to  slow  down  as  one  gets  older.  Second,  the  older  pilots  learned  to  fly 
when  HUDs  were  less  common,  and  they  lack  the  video  game  experience  that  is  actually 
valuable  when  flying  this  type  of  simulator.  Finally,  they  may  have  had  experience  with 
traditional  GCAS  and  had  trouble  adapting  to  new  symbology,  while  the  younger  pilots  had 
never  used  GCAS  and  thus  had  no  paradigms  to  break.  If  the  third  reason  was  substantial  in 
causing  the  performance  decrease,  it  would  suggest  that  any  new  symbology  must  be 
implemented  carefully  with  adequate  training  to  ensure  pilots  learn  to  use  it  well. 

5.2.7  Effort  Measurement 

One  of  the  objectives  of  the  experiment  was  to  examine  the  pilot  effort  level  required  to 
follow  the  various  symbologies.  A  large  amount  of  pilot  effort  suggests  that  an  advanced 
PGCAS  display  is  inefficient  and  perhaps  uncomfortable  to  use.  It  may  also  be  unnatural, 
requiring  constant  inputs  and  corrections  to  null  out  previous  errors.  One  measure  of  pilot  effort 
is  longitudinal  stick  position.  Large  oscillating  stick  inputs  suggest  that  the  pilot  was  over¬ 
compensating  and  was  thus  kept  quite  busy  with  the  altitude  tracking  task.  If  fewer  stick  inputs 
were  recorded,  the  pilot  was  better  able  to  judge  the  correction  needed  to  null  a  given  error. 
Figure  5.14  shows  the  RMS  value  for  longitudinal  stick  position,  averaged  between  runs  and 
between  subjects. 
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Figure  5.14:  RMS  Longitudinal  Stick  Position  (n  =  72) 

One  can  see  that  the  Highway  produced  the  least  longitudinal  stick  input,  although  it  was  not 
significantly  better  than  the  Chevrons.  The  Break-X  again  performed  worse  than  the  other  three 
displays,  because  most  pilots  made  large  inputs  as  they  attempted  to  fly  just  above  the  altitude 
where  the  X  would  appear.  The  significant  difference  between  the  Chevrons  and  Mountain 
coincides  with  pilot  comments  that  the  Mountain  appeared  to  move  faster  and  therefore  required 
more  effort  to  follow.  In  truth,  the  two  symbols  moved  at  the  same  velocity,  but  the  Mountain 
moved  across  a  vertical  space  while  the  Chevrons  moved  horizontaUy.  To  the  human  eye, 
vertical  lines  and  spaces  appear  larger  than  equal-length  horizontal  lines  and  spaces.^^  Since  the 
Mountain  moved  across  an  apparently  larger  space  in  equal  time,  it  may  have  seemed  faster. 
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Interestingly,  the  pilots  had  a  harder  time  following  the  Mountain  but  performed  better  using  it. 
This  suggests  a  trade-off  between  effort  level  and  performance  for  these  two  displays. 

5.3  Subject  Comments 

The  twelve  subjects  were  interviewed  orally  after  using  each  display,  and  the 
experimenter  recorded  their  responses.  The  responses  appear  in  Appendix  D. 

5.3.1  Strategies  and  Preferences 

The  first  question  asked  what  strategies  the  pilots  developed  for  following  the  alerts  and 
which  they  felt  worked  the  best.  According  to  their  responses  and  the  recorded  altitude  profiles, 
the  Break-X  produced  a  great  deal  of  oscillation.  Generally,  the  pilot  would  pull  hard  until  the 
Break-X  disappeared.  He  would  then  lower  the  nose  to  -10”  or  -20°  flightpath  angle  and  wait  for 
the  symbol  to  appear  again.  All  agreed  that  the  Break-X  wastes  energy  and  effort  and  does  not 
provide  much  feedback.  The  Break-X  was  called  effective  for  alerting  the  pilot,  but  very 
ineffective  for  terrain  following  and  terrain  avoidance.  One  subject  thought  that  the  Break-X  was 
better  for  getting  the  pilot’s  attention  than  the  other  alerts.  The  pilots  agreed  that  the  Break-X 
was  the  poorest  alert  for  this  task  because  of  its  binary  nature.  They  were  in  danger  of  ground 
collision  with  the  X  present,  and  in  possible  danger  of  SAM  shootdown  with  the  X  absent;  no 
state  was  considered  safe. 

The  main  comment  about  the  Chevrons  was  the  ambiguity  arising  from  the  horizontal 
orientation  of  the  axes.  Eight  subjects  commented  that  the  horizontal  movement  of  the  chevrons 
did  not  correspond  to  the  vertical  motion  of  the  outside  world.  For  some  people,  this  caused 
confusion  under  duress  about  which  way  the  chevrons  were  desired  to  move.  Quick  movement 
of  the  chevrons  in  either  direction  often  caused  the  pilots  to  pitch  upward,  even  if  the  chevrons 
had  moved  outward  (toward  safety).  This  reverse-control  is  very  dangerous  and  difficult  to 
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overcome  once  it  occurs.  Several  subjects  noted  that  they  only  needed  one  chevron  because  the 
display  was  symmetric  across  the  vertical  axis  and  both  chevrons  gave  the  same  information. 
This  suggests  using  a  single  symbol  such  as  the  Mountain.  The  pilots  appreciated  the  fact  that  the 
Chevrons  moved  out  of  the  FOV  as  danger  decreased.  Thus,  they  could  be  seen  peripherally 
without  interfering  with  the  other  HUD  symbology.  But  when  danger  was  present,  they  moved  to 
center-HUD  and  attracted  attention. 

The  pilots  liked  the  vertical  motion  of  the  Mountain  because  of  the  way  it  corresponded 
to  the  outside  world.  Some  commented  that  it  implied  the  shape  of  the  terrain,  which  was  not 
necessarily  true.  When  compared  with  the  Highway,  the  consensus  was  that  the  Mountain  moved 
more  quickly  with  less  warning.  Thus,  people  found  it  hard  to  keep  the  aircraft  reference  symbol 
near  the  top  of  the  mountain  in  rugged  terrain.  They  also  commented  that  the  Mountain  would 
drop  off  the  screen  above  the  WCH,  leaving  no  indication  of  danger  when  flying  too  high.  If 
danger  still  exists,  it  should  be  shown  by  the  display.  Some  pilots  thought  that  the  mountain  icon 
spreading  across  the  top  of  the  HUD  appeared  to  be  less  urgent  than  a  central  X.  Others 
commented  that  the  Break-X  cluttered  the  center  of  the  HUD  because  it  didn’t  move.  This 

illustrates  the  trade-off  between  urgency  and  clutter  that  must  be  addressed  when  designing  this 
type  of  display. 

Overall,  the  pilots  felt  the  most  comfortable  performing  the  altitude  tracking  task  with  the 
Highway  because  of  its  predictive  nature.  One  positive  aspect  was  its  correspondence  to  the 
outside  world,  which  allows  one  to  think  of  it  as  a  physical  object.  Pilots  tended  to  fly  the  FPM 
along  the  surface  far  ahead  of  the  aircraft,  noting  their  current  altitude  by  the  location  of  the 
nearer  bars.  Eight  subjects  said  that  they  could  anticipate  the  terrain  ahead  much  better  using  the 
Highway,  allowing  them  to  time  control  inputs  to  follow  the  terrain  more  accurately.  Among  the 
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suggestions  for  improving  the  display  were  to  add  exponential  scaling,  to  change  the  post 
spacing  with  the  ruggedness  of  the  terrain,  and  to  thin  out  the  bars  to  demand  less  of  the  HUD 
drawing  computer. 

5.3.2  Rolling  Tendency 

The  second  question  asked  how  the  pilots’  tendency  to  roll  wings-level  changed  between 
the  four  displays.  Because  the  Break-X  was  stationary,  it  provided  no  bank  indication  and  the 
subjects  agreed  that  it  did  not  hinder  or  help  with  rolling  to  wings-level.  The  Chevrons  moved, 
but  they  were  not  intended  to  represent  objects  in  the  outside  world.  Thus,  they  were  still 
considered  icons,  and  the  subjects  seemed  able  to  keep  roll  motion  separate  from  the  pitching 
demanded  by  the  chevrons.  The  Mountain,  however,  had  a  physical  analogue  because  of  its 
vertical  motion.  Thus,  all  of  the  pilots  agreed  that  it  should  be  roll-stabilized.  Some  found  it  to  be 
a  false  indicator  of  wings-level  because  it  appeared  level  even  if  the  aircraft  was  banked. 
Overwhelmingly,  the  Highway  was  preferred  for  keeping  the  pilot  aware  of  bank  angle.  Pilots 
commented  that  it  is  natural  to  view  the  symbology  vertically,  and  one  automatically  rolls  to 
establish  wings-level  so  that  the  surface  moves  up  and  down.  This  corresponds  to  the  analytical 
data,  which  showed  the  Highway  producing  the  least  RMS  roll  error. 

5.3.3  Qualitative  Ranking 

The  third  question  asked  the  subjects  to  qualitatively  rank  the  four  alerts  and  provide 
reasons.  All  of  the  subjects  preferred  the  Highway  overall.  Among  its  listed  advantages  were 
good  roll  indications,  important  predictive  information,  and  improved  SA.  About  half  of  the 
subjects  ranked  the  Chevrons  second,  while  the  others  ranked  it  third.  Those  who  preferred  the 
Mountain  to  the  Chevrons  appreciated  the  way  the  mountain  symbol’s  motion  corresponded  to 
the  terrain.  They  called  the  Chevron  display  “counterintuitive”  because  axis  of  motion  was  not 
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aligned  with  the  physical  world.  All  of  the  subjects  ranked  the  Break-X  last  for  the  tasks  they 
performed  in  this  experiment.  It  was  considered  effective  for  getting  the  pilot’s  attention  and 

prompting  the  initial  pullup  but  ineffective  for  providing  roll  information  and  guidance  during 
the  remainder  of  the  recovery. 

5.3.4  Comments  on  Simulation 

The  last  question  asked  for  general  comments  on  the  simulation.  All  subjects  felt  they 
had  adequate  training  time  and  sufficient  rest  between  tasks.  Most  claimed  they  were  not 
expectmg  the  alert  when  it  arrived,  which  meant  the  goal  of  surprise  was  achieved.  Some  felt 
they  improved  across  the  six  runs  with  a  given  display  or  across  the  four  displays.  This  was 
primarily  due  to  improved  knowledge  of  the  aircraft  dynamics.  Several  improvements  were 
suggested  for  the  various  displays,  but  the  simulation  itself  was  well-accepted. 

5.4  Analytical  Hierarchy  Process 

The  subjective  comparison  ratings  were  used  with  the  Analytical  Hierarchy  Process 
(AHP)  to  obtain  overall  display  preferences  for  each  subject  in  terms  of  percentages.^^  The  first 
question  asked  the  subjects  to  compare  how  much  the  four  displays  helped  them  during  the 
mitial  pull-up.  This  is  often  the  most  crucial  part  of  the  recovery  since  the  aircraft  is  in 
immediate  danger  of  collision.  The  subjects  were  asked  to  separate  the  initial  pull-up  fi-om  the 
remainder  of  the  recovery,  and  the  results  are  shown  in  Figure  5.15. 
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Figure  5.15:  Initial  Pull-up  Usefulness  (n  =  12) 

As  shown  by  the  pie  chart,  the  subjects  considered  the  Highway  the  most  useful,  followed  by  the 
Mountain,  the  Chevrons,  and  the  Break-X.  The  Highway  was  preferred  about  twice  as  much  as 
the  Mountain,  three  times  as  much  as  the  Chevrons,  and  five  times  as  much  as  the  Break-X.  The 
Break-X  appeared  in  the  center  of  the  pilot’s  field  of  view  and  was  the  only  HUD  symbology 
that  did  not  move.  This  made  it  stand  out  from  other  HUD  objects  and  alerted  the  pilot  of 
impending  danger.  Because  it  was  binary,  though,  it  gave  no  indication  of  how  much  danger 
existed;  only  that  it  did  exist.  The  second  question  asked  how  much  the  four  displays  helped  with 
maintaining  the  Target  Clearance  Height  after  the  initial  pull-up.  This  was  to  evaluate  the  terrain¬ 
tracking  portion  of  the  task,  and  the  results  are  shown  in  Figure  5.16 
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Figure  5.16:  Terrain  Following  Usefulness  (n  =  12) 

The  figure  shows  the  same  order  of  preference  as  the  first  question,  but  the  Break-X  falls  further 
behind  the  other  displays.  This  is  likely  because  it  was  effective  initially  for  getting  the  pilot’s 
attention,  but  it  became  ineffective  as  a  terrain  following  cue.  In  Figure  5.17,  one  can  see  the 

results  fi’om  the  third  question,  which  asked  the  subjects  to  rate  their  overall  preference  of  the 
four  displays. 
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Figure  5.17:  Overall  Display  Preference  (n  =  12) 

Again,  the  Break-X  is  far  behind  the  other  three  displays.  It  lacks  the  terrain  closure-rate 
information  that  is  present  in  the  others,  making  it  a  very  uncomfortable  display  to  use  for  this 
task.  The  Chevrons  and  Mountain  were  ranked  approximately  evenly,  although  each  subject 
tended  to  prefer  one  or  the  other.  The  Highway  was  preferred  about  three  times  as  much  as  either 
of  them,  and  its  primary  advantage  was  that  it  showed  upcoming  terrain. 


89 


90 


6  Conclusions  and  Recommendati  ons 


Four  advanced  ground  collision  avoidance  displays  were  tested  using  a  fixed-base  T-38 
simulator  with  a  projection  screen  and  simulated  HUD.  Twelve  subjects  flew  through  a  series  of 
PGCAS  situations  in  zero  visibility  using  each  display.  For  about  30  seconds  after  the  alert,  they 
attempted  to  maintain  a  set  AGL  altitude,  clear  of  the  terrain  but  below  ground  radar  threats. 

The  Break-X  display  was  similar  to  the  standard  military  alert  except  that  it  remained  on 
the  screen  until  the  aircraft  was  recovered  to  safety.  The  Chevron  display  slid  horizontally 
inward  as  danger  increased,  forming  an  X  identical  to  the  break-X.  The  Mountain  display 
utilized  a  terrain  icon  which  moved  upward  in  the  HUD  as  danger  increased,  mimicking  the 
motion  of  the  terrain  outside.  The  Highway  display  was  a  preview  depiction  of  a  perspective 
elevated  surface  at  the  desired  altitude. 

6.1  Conclusions 

6.1.1  Break-X 

The  experimental  results  show  that  the  Break-X  display  performed  significantly  worse 
than  the  other  three  displays.  First,  it  produced  more  RMS  roll  error  than  any  of  the  other 
displays.  It  was  immediately  urgent,  so  pilots  tended  to  pull  immediately  instead  of  first 
evaluating  the  aircraft  attitude  and  rolling  appropriately.  Second,  the  Break-X  provided  little 
security  and  produced  poor  altitude  following.  When  given  a  Break-X,  subjects  spent  significant 
time  outside  the  bounds  within  which  they  were  instructed  to  fly.  Rough  terrain  further  increased 
the  performance  gap  between  the  Break-X  and  the  three  dynamic  displays.  The  Break-X  allowed 
more  than  twice  as  many  crashes  and  Warning  Clearance  Height  (WCH)  penetrations  as  the  next 
best  display.  Third,  pilot  effort  was  the  highest  with  the  Break-X.  Subjectively,  it  placed  a  distant 
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last  behind  the  other  three  displays.  The  subjects  considered  it  effective  for  alerting  the  pilot  but 

very  impractical  for  a  terrain-foUowing,  terrain-avoidance  (TF/TA)  task  like  the  one  performed 
here. 

6.1.2  Chevrons 

The  Chevron  display  was  significantly  better  than  the  Break-X  in  all  areas.  It  produced 
much  less  roll  error  and  permitted  more  accurate  altitude  tracking.  Using  the  Chevrons,  pilots 
spent  about  70%  of  the  flight  time  between  the  desired  clearance  heights,  compared  to  40%  for 
the  Break-X.  The  Chevron  display  allowed  only  half  as  many  crashes  and  ceiling  penetrations  as 
the  Break-X,  meaning  it  is  much  better  for  terrain  avoidance  and  leveling  off  after  the  initial 
pull-up.  Average  altitude  error  was  only  half  what  the  pilots  accumulated  when  given  the  Break- 
X.  The  Chevrons  produced  less  stick  movement  than  the  Break-X  or  Mountain.  Their  horizontal 
motion  may  have  appeared  slower  than  the  vertical  Mountain  motion,  so  they  prompted  fewer 
error  corrections.  About  half  the  subjects  preferred  the  Chevrons  to  the  Mountain,  while  the 
others  ranked  the  Mountain  higher.  The  chief  complaint  about  the  Chevrons  was  the  fact  that 
their  horizontal  motion  had  no  physical  meaning  in  the  real  world.  This  sometimes  led  to 
ambiguity  and  even  reverse-control. 

6.1.3  Mountain 

The  Mountain  produced  shghtly  more  roll  error  than  the  Chevrons,  probably  because  it 
provided  the  false  illusion  of  wings-level.  This  led  to  entire  recoveries  performed  at  a  significant 
bank  angle  and  therefore  larger  RMS  roll  errors.  This  problem  would  be  alleviated  with  roll- 
stabilization,  that  is,  rotating  the  icon  to  conform  with  the  horizon.  Compared  to  the  Chevrons, 
the  Mountain  resulted  in  similar  but  slightly  better  altitude  performance.  PUots  spent  slightly 
more  time  between  the  desired  clearance  heights  and  less  time  above  the  tactical  ceiling. 
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However,  they  crashed  about  as  many  times  using  the  Mountain  as  they  did  using  the  Chevrons. 
Interestingly,  the  improved  altitude  performance  came  at  the  cost  of  significantly  more  stick 
movement.  A  possible  cause  is  that  the  Mountain  was  perceived  to  move  more  quickly  than  the 
Chevrons.  The  primary  objection  to  the  Mountain  was  that  it  was  not  horizontally  stabilized,  so  it 
provided  the  false  illusion  of  wings-level. 

6.1.4  Highway 

The  Highway  outperformed  the  other  three  displays  in  every  category.  It  produced  the 
least  roll  error,  although  not  significantly  less  than  the  Mountain  or  Chevrons.  This  is  probably 
because  it  was  roll-stabilized;  meaning  the  pilots  could  draw  both  roll  and  flight  path  information 
from  the  single  HUD  object.  The  Highway  allowed  the  most  flight  time  within  the  desired 
altitude  layer;  about  90%  compared  to  80%  for  the  Mountain  and  70%  for  the  Chevrons.  The 
Highway  resulted  in  the  fewest  crashes  and  ceiling  busts,  and  it  produced  the  least  stick 
movement  although  not  significantly  less  than  the  Chevrons.  The  pilots  made  timelier,  more 
appropriate  inputs  with  the  predictive  information  offered  by  the  perspective  surface.  The 
Highway  and  Mountain  were  nearly  equal  over  smooth  terrain,  but  the  Highway  was  much  better 
over  rough  terrain.  Subjectively,  the  Highway  was  rated  as  the  most  desirable  display  for  the 
initial  roll,  the  pull-up,  and  the  remainder  of  the  recovery. 

6.2  Recommendations 

6.2.1  Future  Work 

Several  pieces  of  follow-on  research  would  further  validate  these  results  and  continue 
progress  toward  a  better  PGCAS  display.  First,  the  conclusions  obtained  from  this  experiment 
suggest  that  a  dynamic  display  providing  feedback  would  better  serve  the  flying  community  than 
the  static  break-X.  Thus,  further  research  should  concentrate  on  dynamic  PGCAS  symbology  and 
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ways  of  implementing  motion  in  the  HUD.  Three  candidates  were  suggested  here,  and  there  are 
numerous  other  ways  of  showing  terrain  closure  rate  to  the  pilot  and  helping  him  determine 
when  the  aircraft  is  recovered  to  safety. 

Second,  the  displays  discussed  could  be  improved  and  expanded  upon  to  help  determine 
PGCAS  alert  requirements.  The  chevrons  should  be  roll-stabilized  to  provide  concurrent  bank 
information.  Because  the  two  chevrons  moved  symmetrically,  there  is  a  possibility  that  only  one 
is  needed.  This  would  produce  less  HUD  clutter,  allowing  more  space  for  other  symbology.  With 
one  or  both  chevrons,  the  pilot  must  be  given  a  definite  indication  whether  danger  is  increasing 

or  decreasing.  This  would  help  prevent  the  ambiguity  and  reverse-control  that  sometimes 
occurred  using  the  Chevrons  in  this  experiment. 

A  vertically  moving  display  like  the  Mountain  naturally  reduces  the  ambiguity.  It  has 
physical  meaning  attached  to  the  outside  world,  and  pilots  are  conditioned  to  see  rising  terrain  as 
a  sign  of  danger.  The  display  should  use  a  roll-stabilized  icon  or  move  the  icon  along  a  roll- 
stabilized  axis.  An  appropriate  gain  must  be  chosen  for  a  Chevrons-  or  Mountain-type  display  so 
that  the  symbology  moves  quickly  enough  to  attract  attention  but  slowly  enough  to  be  anticipated 
and  followed.  The  shape  of  the  mountain  icon  could  change  to  show  the  characteristics  of  the 
terrain  ahead,  much  like  the  kinematic  horizon  discussed  earlier.  If  this  is  impossible,  it  should 
mamtain  a  generic  shape  that  does  not  imply  real  terrain  shaped  the  same  way. 

To  aUow  even  better  anticipation,  a  preview  display  like  the  Highway  should  be  used. 
One  possible  improvement  would  be  to  scale  vertical  movement  exponentially  instead  of 
linearly.  Although  this  would  prevent  the  display  fi-om  being  completely  conformal,  it  would 
help  prevent  very  rapid  motion  toward  the  extremes  of  the  HUD  and  make  the  display  less 
distracting.  Also,  the  post  spacing  could  be  reduced  or  adapted  to  the  terrain  type  in  order  to 
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reduce  the  number  of  lines.  A  tunnel  display  might  better  serve  the  purpose  of  showing  two 
altitudes,  such  as  the  SCH  floor  and  the  WCH  ceiling.  In  this  simulation,  some  pilots  would  have 
preferred  flying  inside  a  corridor  rather  than  along  a  surface.  Figure  6. 1  shows  one  format  for 
displaying  two  altitudes  along  with  the  terrain. 


Figure  6.1:  Tunnel  Display  Possibility 

The  base  of  the  legs  still  would  represent  the  terrain,  but  in  this  case  a  ceiling  would  show  the 
pilot  the  maximum  desired  altitude.  The  height  of  the  tunnel  could  change  based  on  ground 
threats  and  terrain  masking  available  in  the  area.  A  tunnel  display  would  be  most  useful  for 
commanding  optimal  turning  recoveries  based  on  PC5CAS  evaluation  of  ground  threats,  aircraft 
performance  available,  and  terrain  characteristics. 

All  of  the  displays  could  be  improved  by  incorporating  more  information  than  just  the 
minimum  predicted  clearance.  Also,  a  smoothing  filter  would  make  the  symbology  more  docile 
and  predictable,  eliminating  most  of  the  annoying  jumps  noted  by  the  pilots. 

.  6.2.2  Implementation 

General  guidelines  regarding  implementation  of  advanced  PGCAS  displays  arise  fi'om 
this  project.  First,  using  pilot  acknowledgement  allows  more  informative  symbology,  which 
remains  on  the  HUD  until  the  pilot  chooses  to  remove  it.  A  more  demanding  system  would 
require  the  pilot  to  physically  move  a  switch  or  push  a  button;  an  alternative  system  might 
assume  acknowledgement  once  the  aircraft  has  maintained  a  safe  altitude  for  a  set  period  of  time. 
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Displays  should  not  clutter  the  HUD,  nor  test  the  drawing  capabilities  of  the  image  generator.  A 
built-in  safety  factor  is  a  necessity,  so  it  is  desirable  to  show  a  clearance  height  that  still  allows 
for  some  error  without  catastrophe. 

The  pUots  in  this  experiment  preferred  a  roll  stabilized,  dynamic  display  that  provided 
preview  information.  Granted,  the  limited-visibility  TF/TA  task  examined  here  is  only  one 
situation  when  a  PGCAS  alert  might  occur.  Still,  further  research  would  likely  prove  continuous 
displays  viable  for  visual  recoveries  any  time  accurate  flightpath  control  is  desired. 
Implementation  of  advanced  displays  such  as  these  would  save  tax  dollars  and,  more 
importantly,  the  lives  of  pilots  who  risk  CFIT  each  time  they  fly. 


96 


Appendix  A  -  Pilot  Instructions 


This  experiment  will  test  your  ability  to  recover  a  T-38  aircraft  in  the  face  of  impending  ground 
collision.  The  Head-Up  Display  is  your  primary  source  of  flight  information.  You  should  leave 
the  throttle  at  military  power  for  the  duration  of  the  experiment,  and  do  not  use  the  trim  switch  or 
flaps. 

Your  task  will  be  to  follow  a  series  of  flight  director  cues  under  instrument  meteorological 
conditions.  These  cues  will  simulate  air  traffic  control  directives  or  terrain  following  commands 
you  might  receive.  You  should  pitch  the  aircraft  to  put  your  velocity  vector  on  the  black 
command  symbol.  You  should  then  match  your  bank  angle  with  that  shown  by  the  command 
symbol.  You  will  not  be  scored  on  how  long  it  takes  to  reach  the  command,  so  take  sufficient 
time  to  do  it  carefully  without  large  oscillations. 


At  some  point  in  the  flight,  you  will  receive  one  of  four  possible  ground  collision  alerts.  The 
alert  will  occur  when  your  projected  trajectory  crosses  the  Warning  Clearance  Height  (WCH). 
You  should  not  penetrate  the  Safety  Clearance  Height  (SCH).  Upon  receiving  an  alert,  you  will 
use  the  standard  military  recovery:  first  rolling  to  wings-level,  and  then  pulling  up.  Recover  to 
the  Target  Clearance  Height  (TCH),  and  attempt  to  maintain  this  separation  until  the  simulation 
is  paused.  The  idea  here  is  to  avoid  detection  by  ground  radar  and  to  prevent  excessive  energy 
loss. 


WCH(750ftAGL) 
TCH  (SOOftAGL) 
SCH  (250ftAGL) 
Terrain 


The  following  table  summarizes  the  four  alerts  you  will  encounter. 


Name: 

Break-X 

Chevrons 

Mountain 

Diagram: 

X 

X 

W 

W 

/r~K 

IIU 

\ 

y 

Aircraft  at  WCH: 

Not  present 

First  appears 

First  appears 

First  appears 

Aircraft  at  TCH: 

First  appears 

Touching  bars 

Touching  A/C 
reference  W 

FP  Marker  level 

with  surface 

Aircraft  at  SCH: 

Present 

FormX 

Touches  top  of 
HUD 

FP  Marker  well 
below  surface 

Aircraft  at 

Terrain: 

Present 

FormX 
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Appendix  B  -  Informed  Consent 

Experimental  Study  of  Head-Up  Display  Symbology 
For  Aircraft  ^ound  Collision  Avoidanee 


Research  Assistant: 
Geoffirey  O.  Billingsley 
Room  6228 

C.  S.  Draper  Laboratory 
Cambridge,  MA  02139 


Principal  Investigator: 
Prof.  James  Kuchar 
MITRm.  33-117 
77  Massachusetts  Ave. 
Cambridge,  MA  02139 


Participation  in  this  study  is  voluntary  and  you  may  stop  or  delay  the  experiment  at  any  time. 
You  may  also  withdraw  from  the  study  for  any  reason,  without  prejudice.  If  you  have  any 
questions  concerning  the  purpose,  procedures,  or  risks  associated  with  this  experiment,  please 
ask  them. 


This  study  is  designed  to  evaluate  several  candidate  alerting  displays  for  terrain  avoidance  during 
low-level  maneuvering.  You  will  be  flying  a  T-38  flight  simulator  and  looking  at  a  screen  that 
shows  the  view  fi'om  the  cockpit.  During  these  flights,  your  task  is  to  respond  to  terrain  alerts  hy 
recovering  the  aircraft.  An  interview  following  the  experiment  may  be  audio  taped  to  aid  in 
recording  conraients.  The  tapes  will  contain  no  identifying  information,  and  the  experimenter 
will  store  them  securely  after  the  experiment.  They  will  be  destroyed  within  one  year  once 
conclusions  have  been  drawn  from  the  data.  The  study  is  expeeted  to  take  a  total  of  2  hours  to 
complete. 

As  with  any  flight  simulation,  you  may  or  may  not  experience  “simulator  sickness,”  ineluding 
disorientation,  vertigo,  dizziness,  headache,  eyestrain,  fatigue,  nausea,  and  in  extreme  cases, 
vomiting.  You  will  continually  be  in  voice  contact  with  the  experimenter,  who  will  be  stationed 
next  to  the  simulator  cockpit,  and  frequent  rest  periods  will  be  provided.  Please  inform  the 
experimenter  at  the  first  sign  of  any  uncomfortable  symptoms,  and  the  experiment  will  be 
interrupted  and  an  attempt  made  to  alleviate  the  eause  of  the  symptoms. 

All  data  will  collected  in  a  confidential  manner  and  will  not  he  linked  in  any  way  to  your 
identity.  You  will  remain  anonymous  in  any  report  that  describes  this  work. 
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CONSENT 


In  the  unlikely  event  of  physical  injury  resulting  from  participation  in  this  research,  I  understand 
that  medical  treatment  will  be  available  from  the  M.I.T.  Medical  Department,  including  first  aid, 
emergency  treatment  and  follow-up  care  as  needed,  and  that  my  insurance  carrier  may  be  billed 
for  the  cost  of  such  treatment.  However,  no  compensation  can  be  provided  for  medical  care  apart 
from  the  foregoing.  I  forther  understand  that  making  such  medical  treatment  available,  or 
providirig  it,  does  not  imply  that  such  injury  is  the  Investigator's  fault.  I  also  understand  that  by 
my  participation  in  this  study  I  am  not  waiving  any  of  my  legal  rights. 

I  understand  that  I  may  also  contact  the  Chairman  of  the  Committee  on  the  Use  of  Humans  as 
Experimental  Subjects,  M.I.T.  253-6787,  if  I  feel  I  have  been  treated  unfairly  as  a  subject. 

I  volunteer  to  participate  in  this  experiment  which  is  to  involve  flight  simulation.  I  understand 
that  I  may  discontinue  my  participation  at  any  time.  I  have  been  informed  as  to  the  nature  of  this 
experiment  and  the  risks  involved,  and  agree  to  participate  in  the  experiment. 


Signature 

Further  information  may  be  obtained  by  calling  the  Institute's  Insurance  and  Legal  Affairs  Office  at  253-2822 
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Appendix  C  -  Pilot  Questionnaire 

Participation  in  this  study  is  voluntary.  You  are  free  to  decline  to  answer  any  question(s)  or 
withdraw  from  the  study  at  any  time,  without  prejudice.  Whether  or  not  you  complete  the 
experiment,  your  confidentiality  and  anonymity  are  assured. 

Personal  Data: 

Age: 

Gender:  M  /  F 

Please  list  your  pilot  experience: 

Total  Hours: 

Ratings: 

Primary  A/C: 

Jet  Experience: 

Military  Hight  Experience: 

HUD  Experience: 

Ground  Collision  Avoidance  System  (GCAS)  Experience: 

What  strategies  did  you  develop  for  following  the  alerts?  Which  do  you  think  worked  the  best? 
Did  you  encounter  differences  in  your  tendency  to  roll  to  wings-level  before  pulling? 

Which  alert  did  you  prefer  and  why? 

Do  you  have  comments  on  the  overall  simulation?  Fatigue?  Rest  time?  Enough  practice? 
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Which  alert  better  helped  you  to  avoid  penetrating  the  lower  Safety  Clearance  Height 
during  the  initial  recovery? 


Break-X 


absolutely  much 

better  better 


Chevrons 


absolutely 

better 


Break-X 


absolutely  much 

better  better 


Mountain 


absolutely 

better 


Break-X 


absolutely  much 

better  better 


Highway 


absolutely 

better 


Chevrons 


absolutely  much 

better  better 


slightly 

better 


Mountain 


absolutely 

better 


Chevrons 


absolutely  much 

better  better 


slightly 

better 


Highway 


absolutely 

better 


Mountain 


absolutely  much 

better  better 


Highway 


absolutely 

better 


102 


Which  alert  better  helped  you  to  maintain  the  Target  Clearance  Height  after  the  initial 
recovery? 


Chevrons 


Break-X 


absolutely  much  better  slightly  same  slightly  better  much  absolutely 

better  better  better  better  better  better 


Break-X 


Mountain 


absolutely  much  better  slightly  same  slightly  better  much  absolutely 

better  better  better  better  better  better 


Break-X 

Highway 

absolutely 

better 

much 

better 

better 

slightly 

better 

same 

slightly 

better 

better 

much 

better 

absolutely 

better 

Mountain 


Chevrons 


absolutely  much  better  slightly  same  slightly  better  much  absolutely 

better  better  better  better  better  better 


Chevrons 

Highway 

absolutely 

better 

much 

better 

better 

slightly 

better 

same 

slightly 

better 

better 

much 

better 

absolutely 

better 

Mountain 

Highway 

absolutely 

better 

much 

better 

better 

slightly 

better 

same 

slightly 

better 

better 

much 

better 

absolutely 

better 
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Overall,  which  alert  did  you  prefer? 


absolutely  much 

better  better 


Break-X 


absolutely 

better 


Highway 


absolutely  much 

better  better 


Chevrons 


^tter  slightly  same 

better 


absolutely 

better 


Mountain 


absolutely  much 

better  better 


Chevrons 


absolutely 

better 


Highway 


absolutely  much 

better  better 


absolutely 

better 


Mountain 


-  _ Highway 


absolutely  much 

better  better 


better  slightly  same 

better 


absolutely 

better 
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Appendix  D  -  Pilot  Comments 

What  strategies  did  you  develop  for  following  the  alerts?  Which  do  you  think  worked  the 

best? 

1)  Break-X:  not  very  helpful. 

2)  Break-X:  Oscillated  a  lot,  pull  hard  then  descend  slowly  using  10°-15"  degree  increments. 
Chevrons:  aggravating,  motion  is  perpendicular  to  reality,  paid  more  attention  to  pitch 
ladder.  Mountain:  covered  pitch  ladder,  so  used  it  less.  Highway:  can  use  it  inverted,  put 
FPM  on  top  of  next  hill,  not  valleys  (would  tend  high) 

3)  Break-X:  Put  FPM  just  below  horizon,  wait  for  X.  Can’t  be  too  effective  for  TF,  but  TA  is 
good.  Chevrons:  pulling  vertical  but  markers  are  horizontal.  Looked  only  at  half  of  it  -  no 
need  for  both.  Pull  or  push?  Tracking  roll  and  flight  path  simultaneously  is  hard.  Mountain: 
No  info  on  what’s  ahead,  coming.  Lost  track  of  bank,  little  hard  to  use  W  vs.  FPM.  Tried  to 
think  ahead  of  it,  predict  what  it  would  do.  Highway:  hard  to  follow  when  terrain  changes 
rapidly,  which  line  am  I  at?  Think  of  it  as  ground,  oscillates  with  large  terrain  changes.  Could 
disappear. .  .where  is  it?  Dotted  on  bottom? 

4)  Break-X:  Noted  altitude  where  alert  happens,  fly  it  more  conservatively.  Let  down 
gradually,  -10°  FPA.  Roll,  then  pull.  More  aggressive  inputs,  more  confidence  since  had  it 
last.  Chevrons:  info  outside  field  of  view,  started  scanning,  then  could  take  them  in 
peripherally.  Not  much  displacement  between  them  -  move  vertical  bars  by  scaling. 
Horizontal  -  less  intuitive  than  vertical,  only  used  one  chevron.  Mountain:  show  something 
when  it  goes  off  bottom  of  HUD.  Make  it  roll.  Fly  it  as  a  rate  cue,  not  displacement.  Went  to 
-10°  FPA,  wait  for  it  to  come  up,  otherwise  comes  too  fast.  Exponential  scaling  also  nice 
here.  Highway:  anticipation  helps,  exponential  scaling  would  be  nice  to  keep  it  from 
disappearing.  Interpret  as  projected.  Nice  to  change  post  spacing  with  terrain  ruggedness.  Try 
to  fly  the  farthest  trend... natural,  easy  to  learn.  Nearest  bar  moves  quickly,  whipping. 

5)  Break-X:  not  very  good.  Real  life:  go  very  positive  till  it  disappears,  then  negative. 
Oscillatory,  wastes  energy  and  effort,  not  much  feedback.  Chevrons:  Liked  how  speed  of 
chevrons  showed  rate  of  terrain  closure.  Nor  foreknowledge;  peak  produced  large  movement. 
Instinct  is  to  just  pull.  Mountain:  bad  since  referenced  to  W,  not  IT’M.  Tried  to  line  up  all 
three,  use  FPM  as  predictor  for  location  of  W.  Highway:  match  up  lines,  collapse  them  if 
terrain  is  flat.  Anticipate  valley,  peak,  cut  off  top  and  bottom.  Could  interfere  with  tapes. 

6)  Break-X:  held  constant  negative  FPA  until  it  reappears.  Have  to  pull  until  it  disappears.  For 
alert,  not  for  TF/TA,  impossible  to  complete  mission.  Full  back  initially,  float  down  to  about 
-20°  FPA.  Not  comfortable  at  all.  Chevrons:  Need  more  time  between  appearing  and  SCH, 
moved  more  slowly  than  Mountain.  Mountain:  between  Highway  and  Break-X,  could  be 
useful  in  mild  rolling  terrain,  not  great  for  rugged  terrain.  Quick  movement  from  bottom  to 
top,  no  future  projection.  Highway:  limited  by  number  of  lines  HUD  can  draw.  Can  teU  what 
terrain  ahead  is  doing.  Recovery  inconsistent  -  typically,  just  enough  to  recover;  makes  pilot 
push  -g  often. 

7)  Break-X:  rolled  to  90,  let  nose  drop  through.  Can  push  over,  no  indication  of  negative  g. 
Note  altitude  when  it  appears.  Uneasy  feeling  pulling  down  in  IMC.  Chevrons:  Highway 
better,  guidance  but  not  command.  Here,  no  information,  just  command.  Not  predictive,  no 
planning.  -10°  FPA  works  fine  for  rate  of  indications.  Don’t  like  pushing  over  in  IMC  GCAS 
situation.  Mountain:  shape  implies  terrain  shape,  not  necessarily  true,  distracting.  No 
anticipation  of  when  it  will  go  down  and  away,  easier  to  maintain  steady  state  once  I  got  it. 
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Highway:  feels  safe  just  under  surface,  too  comfortable.  Should  feel  worse  underneath 

Highway  surface.  Hard  to  get  right  on.  Airspeed  and  altitude  dropped  out  of  cross-check 
Operationally,  thin  out  bars. 

8)  Break-X:  don  t  know  what’s  ahead,  mountain  or  flat.  Overcorrect,  err  to  high  side.  -5°  or  - 
10"  FPA  to  get  X  to  reappear.  Chevrons:  can’t  focus  on  HUD  still.  Horizontal  doesn’t  match 
With  vertical,  moved  slower  than  Mountain  (worse),  it  seemed.  Mountain:  tendency  to  chase 
can  t  really  ^ticipate  it.  Could  drop  off  screen,  can’t  follow  it  exactly.  Focus  on  it  vs. 

chevrons.  Highway:  much  better,  can  tell  how  far  above  should  be.  Anticipate  much  better 
could  use  one  full  time  for  TF. 

9)  Br^k-X:  inqiossible  to  track  terrain,  no  prehminary  warning.  -20°  increments  on  ladder  to 
make  X  reappear.  Chevrons:  fine  for  initial  pullup.  If  terrain  moves,  very  difficult.  400-500 
kts  too  fast  for  500  ft  clearance  unless  rolling  terrain.  Ambiguous  if  inverted,  symmetric 
vertically.  Mountain:  fools  you  to  think  you’re  wings  level,  easier  to  use  than  Chevrons. 
Terrain  unrealistic  for  t^  airspeed,  though.  Highway:  best  feedback,  good  roll  info.  Tend  to 
damp  out  bumps,  more  info  when  slightly  below  Highway  surface. 

lOjBi^ak-X:  got  attention  more  than  other  two.  No  good,  have  to  pull  hard  since  don’t  know 
puU  untU  way  high,  then  push  negative  g’s.  Go  to  -20°  FPA,  then  up  to  wait  at  -5°  or  -10°  ’ 
Chevrons:  counterintuitive,  can’t  imagine  a  mountain,  but  can  use  peripheral  vision  more. 

No  mfo  between  250  and  0  ft.  Mountain:  good  because  you  could  see  rate.  Tried  not  to  go 
below  -15°  FPA.  Tried  to  keep  it  on  screen,  get  fixated  on  symbol.  Highway:  very  good  for 

^tter  for  saying  bad,  add  it  to  Highway.  Hy  lower  if  terrain  ahead 
falls  off,  start  pitching  ahead  of  time.  Could  lead  terrain  more. 

11) Break-X:  insulting,  not  intended  for  terrain  tracking,  pure  alert,  binary.  Uncomfortable 
feelmg.  Chevrons:  not  intuitive,  reference  frames  perpendicular.  Training  could  help,  but  in 
times  of  duress  may  still  reverse  control.  This  happened  and  was  not  easy  to  correct.  When 
chevrons  are  gone  or  together,  you  ^e  left  with  no  info,  lost.  Mountain:  axes  aligned,  much 
better.  Lose  sight  of  it,  left  with  no  info.  Purely  reactionary,  no  anticipation.  Flew  to  -20° 
PTA,  waited  for  it.  Felt  like  my  gain  was  higher  for  this  display.  Highway:  best  performance, 
reference  frames  intuitive.  No  distance-rate  cueing,  inner  loop  control,  so  I  knew  what  to  do 
in  next  10  seconds  but  not  during  next  1  second.  Can  anticipate  some,  but  not  enough. 

12)  Break-X:  couldn  t  stand  it.  No  place  was  ok  -  either  emergency  or  too  high. 

Counterintuitive,  to  be  ok,  must  put  self  in  danger.  Chevrons:  hard  to  track  when  they 
disappear.  Not  intuitive  to  pull  up  because  axes  were  perpendicular.  Quick  movement  meant 
pull,  even  if  movement  was  outward  instead  of  inward  (reverse  control).  Meeting  seemed 
very  bad,  worse  than  Mountain  flattening,  centered  in  middle  of  screen.  Mountain:  pretty 
good,  felt  like  doing  well.  Bothers  that  symbol  is  always  level.  Concentrating  on  tracking, 
forgot  about  roll.  Highway:  feels  less  stressful,  less  workload.  Like  it  a  lot,  very  easy  to  slack 

off,  cut  off  bottom  of  valleys.  Felt  like  I  was  doing  something  versus  being  commanded 
around. 

Did  you  encounter  differences  in  your  tendency  to  roll  to  wings-level  before  pulling? 

2)  Highway  best  for  roll  indicator 

3)  Highway  very  nice  for  rolling  to  wings-level.  Chevrons  and  other  two  much  more  difficult. 
Would  need  to  be  a  habit. 
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4)  Mountain:  Was  used  to  other  one  rolling,  didn’t  think  about  it  till  end.  Rolling  alert  allows 
quick  assimilation  of  data. 

5)  Highway  was  good  for  helping  recover,  gave  roll  information.  Break-X  not  as  good  as 
Highway.  Chevrons  and  Mountain  could  be  rotated. 

6)  Experience  helped  with  rolling  first,  all  should  be  roll  stabilized. 

7)  Break-X:  Some  loaded  rolls,  some  unloaded.  Instinct  calls  for  loaded  roll.  Highway:  pause 
between  command  and  Highway  appearing,  still  headed  down  because  knew  ground  was 
coming.  Chevrons:  preferred  over  Mountain;  more  anticipation.  Mountain:  gives  illusion  of 
wings-level,  vertical  orientation. 

8)  Break-X:  forgot  once.  Highway:  more  cues  of  wings-level.  Mountain:  false  illusion  that 
wings  are  level,  translates  to  physical  cue.  Chevrons:  Easier  to  remember  to  roll  first,  iconic. 

9)  Mountain:  worse  than  Chevrons,  false  illusion  of  wings-level.  Break-X:  better  for  roll. 
Highway:  best,  very  natural. 

10)  Chevrons,  Mountain,  Break-X  about  the  same.  Highway  much  better. 

11)  Chevrons,  Mountain  gave  no  intuitive  cues  for  rolling,  override  world  reference  frame.  Better 
to  roll  stabilize  them  Highway:  much  better  for  rolling  to  wings-level,  natural  to  view  it  in 
vertical,  try  to  roll  until  it  is  vertical. 

12)  Mountain:  Remembered  to  roll  initially,  forgot  after  that  sometimes.  Chevrons:  about  same 
as  Mountain,  both  fixed.  Highway:  roll-stabilized,  inakes  a  big  difference,  more  immersive. 
Looks  like  ground,  not  sure  what  it  represents.  Break-X:  same  as  Chevrons,  Mountain,  but  a 
little  better  because  not  concentrating  on  tracking. 

Which  alert  did  you  prefer  and  why? 

1)  Either  the  Chevrons  or  Highway:  using  the  Highway  it  was  hard  to  keep  the  TCH.  Using  the 
Chevrons  it  was  the  easiest  to  keep  the  TCH.  Tempted  to  fly  inside  the  Highway,  feel  lost 
above  it.  Mountain  pretty  good  also. 

2)  Highway:  good  roll  indicator,  shows  info  ahead,  what  is  coming. 

3)  Highway:  modifications  would  be  useful.  Predictive  -  sharp/dull  pullups.  Could  “see”  terrain. 

4)  Highway:  fly  it  as  trend,  flight  cue. 

5)  Highway:  gives  info  ahead  of  time.  Flat  terrain:  any  but  Break-X.  Peaky  terrain:  hard  to  keep 
up  with  Chevrons,  Mountain. 

6)  Highway:  more  S  A,  more  information  presented  than  current  or  set  distance. 

7)  Highway,  then  Chevrons,  then  Mountain,  then  Break-X. 

8)  Highway,  then  Chevrons,  then  Mountain,  then  Break-X. 

9)  Highway  best  overall.  Chevrons  were  counterintuitive,  no  roll  information. 

10)  Highway,  then  Mountain,  then  Chevrons,  then  Break-X. 

11)  Highway,  then  Mountain,  then  Chevrons,  then  Break-X. 

12)  Highway,  then  Mountain,  then  Chevrons,  then  Break-X 

Do  you  have  comments  on  the  overall  simulation?  Fatigue?  Rest  time?  Enough  practice? 

1)  OK 

2)  Sometimes  knew  when  alert  was  coming.  Some  fatigue  near  end,  possibly  learning 

3)  Possibly  small  improvement  through  simulation,  sitting  in  sim  is  best  training. 

4)  Fly  to  vs.  fly  from  on  command  symbol,  confusing.  Time  to  Impact  vs.  Altitude  to  Impact. 
Mountain  -  noise  at  about  1  V2  Hz. 
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5)  Didn  t  feel  like  same  terrain  for  all  four,  couldn’t  remember  when  alert  was  coming 
probably  improved  through  runs,  not  through  alerts. 

6)  Plenty  of  practice.  Break-X  is  hardest.  Highway  easiest.  No  improvement  through  runs. 

)  Highway,  great  knowing  terrain  ahead,  show  ground,  SCH,  maximum  desired  altitude 

8)  Earning  through  alerts,  also  through  runs.  Could  tell  by  memory  that  mountain  is  coming, 
w  task  to  use  sign  of  impending  danger  to  track,  confusion  between  X  and  tracking. 

Would  want  Highway  poles  to  move  past,  show  10-20  seconds  ahead.  Make  it  uncomfortable 
to  crash,  warn  more  against  it. 

10)  Trimmed  once.  Sim  was  realistic,  appeared  random.  Learning  through  runs.  Plenty  of 
practice,  some  learning  about  dynamics. 

1 1)  Much  more  confident  with  Mountain  than  Chevrons,  could  be  learning.  First  3-4  runs,  still 
getting  used  to  d5niamics.  No  fatigue,  boredom,  timespan  ok. 

12)  Terrain  pretty  crazy.  First  rung  of  Highway  unattached.  Could  put  it  in  bold,  change 
Highway  to  different  color  when  underneath  surface.  Cluttered  display. 
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